Re: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the stack filters

2021-08-09 Thread Xiang, Haihao 
On Sat, 2021-08-07 at 03:24 +, Soft Works wrote:
> > -Original Message-
> > From: ffmpeg-devel  On Behalf Of
> > Xiang, Haihao
> > Sent: Friday, 6 August 2021 07:15
> > To: ffmpeg-devel@ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the
> > stack filters
> > 
> > On Thu, 2021-08-05 at 15:53 +, Soft Works wrote:
> > > > -Original Message-
> > > > From: ffmpeg-devel  On Behalf Of
> > > > Xiang, Haihao
> > > > Sent: Thursday, 5 August 2021 04:33
> > > > To: ffmpeg-devel@ffmpeg.org
> > > > Subject: Re: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of
> > > > the stack filters
> > > > 
> > > > On Wed, 2021-08-04 at 09:17 +, Soft Works wrote:
> > > > > > -Original Message-
> > > > > > From: ffmpeg-devel  On
> > 
> > Behalf
> > > > > > Of Haihao Xiang
> > > > > > Sent: Wednesday, 4 August 2021 10:33
> > > > > > To: ffmpeg-devel@ffmpeg.org
> > > > > > Cc: Haihao Xiang 
> > > > > > Subject: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of
> > > > > > the stack filters
> > > > > > 
> > > > > > Include hstack_qsv, vstack_qsv and xstack_qsv, some code is copy
> > > > > > and pasted from other filters
> > > > > > 
> > > > > > Example:
> > > > > > $> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265
> > > > > > -filter_complex "[0:v][0:v]hstack_qsv" -f null -
> > > > > > ---
> > > > > 
> > > > > [...]
> > > > > 
> > > > > > +
> > > > > > +/*
> > > > > > + * Callback for qsvvpp
> > > > > > + * @Note: qsvvpp composition does not generate PTS for result
> > 
> > frame.
> > > > > > + *so we assign the PTS from framesync to the output frame.
> > > > > > + */
> > > > > > +
> > > > > > +static int filter_callback(AVFilterLink *outlink, AVFrame *frame) {
> > > > > > +QSVStackContext *sctx = outlink->src->priv;
> > > > > > +
> > > > > > +frame->pts = av_rescale_q(sctx->fs.pts,
> > > > > > +  sctx->fs.time_base, outlink-
> > > > > > >time_base);
> > > > > > +return ff_filter_frame(outlink, frame); }
> > > > > 
> > > > > If the surface.Data.TimeStamp gets overwritten by libMFX, why not
> > > > > copy the PTS from the input frame in ff_qsvvpp_filter_frame ?
> > > > > 
> > > > > That would apply the timestamp from the last input, though.
> > > > > Preferably would it be taken from the first input instead. For
> > > > > 2-n, you could perhaps clone the frames and assign the pts from
> > > > > the first input's frame?
> > > > 
> > > > Thanks for the comment and suggestion. This callback function was
> > > > copy-
> > > > and- pasted from overlay_qsv filter because MSDK composition is also
> > > > used by this filter, I'd like to use the same way to generate pts
> > > > for these filters, but I'll try your suggestion for these filters in
> > > > the future.
> > > 
> > > Yea I see - the overlay_qsv filter does it the same way. This has
> > > probably been ok earlier because the callback happened synchronously.
> > > This is no longer the case since the async_depth patch which
> > > introduced the fifo processing. Now it can happen that the calback is
> > > performed for an earlier frame than the one that is currently gated by
> > > framesync.
> > 
> > async_depth is not enabled for overlay_qsv and stack qsv filters, s-
> > > async_depth is 0, so the callback is still performed synchronously for
> > > these
> > 
> > filters.
> 
> Yes I know. But with the newly added looping and read/write from the
> fifos, I'm not sure whether it's always guaranteed that the callback will be
> called for the submitted frame or whether it could happen that there's 
> another out_frame left in the fifo.

I think it is guaranteed because there is at most 1 qsv fifo item in the fifo
for these filters. 

> 
> Another detail that doesn't look solid to me is the acquisition of the
> out_frame
> in cases when compo

Re: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the stack filters

2021-08-06 Thread Soft Works 



> -Original Message-
> From: ffmpeg-devel  On Behalf Of
> Xiang, Haihao
> Sent: Friday, 6 August 2021 07:15
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the
> stack filters
> 
> On Thu, 2021-08-05 at 15:53 +, Soft Works wrote:
> > > -Original Message-
> > > From: ffmpeg-devel  On Behalf Of
> > > Xiang, Haihao
> > > Sent: Thursday, 5 August 2021 04:33
> > > To: ffmpeg-devel@ffmpeg.org
> > > Subject: Re: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of
> > > the stack filters
> > >
> > > On Wed, 2021-08-04 at 09:17 +, Soft Works wrote:
> > > > > -Original Message-
> > > > > From: ffmpeg-devel  On
> Behalf
> > > > > Of Haihao Xiang
> > > > > Sent: Wednesday, 4 August 2021 10:33
> > > > > To: ffmpeg-devel@ffmpeg.org
> > > > > Cc: Haihao Xiang 
> > > > > Subject: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of
> > > > > the stack filters
> > > > >
> > > > > Include hstack_qsv, vstack_qsv and xstack_qsv, some code is copy
> > > > > and pasted from other filters
> > > > >
> > > > > Example:
> > > > > $> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265
> > > > > -filter_complex "[0:v][0:v]hstack_qsv" -f null -
> > > > > ---
> > > >
> > > > [...]
> > > >
> > > > > +
> > > > > +/*
> > > > > + * Callback for qsvvpp
> > > > > + * @Note: qsvvpp composition does not generate PTS for result
> frame.
> > > > > + *so we assign the PTS from framesync to the output frame.
> > > > > + */
> > > > > +
> > > > > +static int filter_callback(AVFilterLink *outlink, AVFrame *frame) {
> > > > > +QSVStackContext *sctx = outlink->src->priv;
> > > > > +
> > > > > +frame->pts = av_rescale_q(sctx->fs.pts,
> > > > > +  sctx->fs.time_base, 
> > > > > outlink->time_base);
> > > > > +return ff_filter_frame(outlink, frame); }
> > > >
> > > > If the surface.Data.TimeStamp gets overwritten by libMFX, why not
> > > > copy the PTS from the input frame in ff_qsvvpp_filter_frame ?
> > > >
> > > > That would apply the timestamp from the last input, though.
> > > > Preferably would it be taken from the first input instead. For
> > > > 2-n, you could perhaps clone the frames and assign the pts from
> > > > the first input's frame?
> > >
> > > Thanks for the comment and suggestion. This callback function was
> > > copy-
> > > and- pasted from overlay_qsv filter because MSDK composition is also
> > > used by this filter, I'd like to use the same way to generate pts
> > > for these filters, but I'll try your suggestion for these filters in
> > > the future.
> >
> > Yea I see - the overlay_qsv filter does it the same way. This has
> > probably been ok earlier because the callback happened synchronously.
> > This is no longer the case since the async_depth patch which
> > introduced the fifo processing. Now it can happen that the calback is
> > performed for an earlier frame than the one that is currently gated by
> > framesync.
> 
> async_depth is not enabled for overlay_qsv and stack qsv filters, s-
> >async_depth is 0, so the callback is still performed synchronously for these
> filters.

Yes I know. But with the newly added looping and read/write from the
fifos, I'm not sure whether it's always guaranteed that the callback will be
called for the submitted frame or whether it could happen that there's 
another out_frame left in the fifo.

Another detail that doesn't look solid to me is the acquisition of the out_frame
in cases when composition is used, i.e. ff_qsvvpp_filter_frame is called
multiple times for the same output frame:

- When VPP returns MFX_ERR_MORE_DATA (awaiting another input for
  overlay), we return without caring about the out_frame (no storing, no
  increasing of the queued property)
- Instead, each time, the out_frame is queried again via query_frame
  The code doesn't really make sure that it operates on the same frame 
  for output, this seems more like a coincidence at the current state
  (probably only the check for surface->Data->Locked in clear_unused_frames
  is preventing the output frame from being cleared meanwhile)

I think there's some room for improvement at least.. 

softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the stack filters

2021-08-05 Thread Xiang, Haihao 
On Fri, 2021-08-06 at 13:17 +0800, Haihao Xiang wrote:
> Include hstack_qsv, vstack_qsv and xstack_qsv, some code is copy and
> pasted from other filters
> 
> Example:
> $> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -filter_complex
> "[0:v][0:v]hstack_qsv" -f null -
> ---
> v2: fix wrong reference in filters.texi

It is v3, not v2 :-(

Thanks
Haihao



___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the stack filters

2021-08-05 Thread Haihao Xiang 
Include hstack_qsv, vstack_qsv and xstack_qsv, some code is copy and
pasted from other filters

Example:
$> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -filter_complex
"[0:v][0:v]hstack_qsv" -f null -
---
v2: fix wrong reference in filters.texi

 configure  |   6 +
 doc/filters.texi   |  81 ++
 libavfilter/Makefile   |   3 +
 libavfilter/allfilters.c   |   3 +
 libavfilter/vf_stack_qsv.c | 498 +
 5 files changed, 591 insertions(+)
 create mode 100644 libavfilter/vf_stack_qsv.c

diff --git a/configure b/configure
index f9fdf58bc3..96c95ac062 100755
--- a/configure
+++ b/configure
@@ -3696,6 +3696,12 @@ vpp_qsv_filter_select="qsvvpp"
 xfade_opencl_filter_deps="opencl"
 yadif_cuda_filter_deps="ffnvcodec"
 yadif_cuda_filter_deps_any="cuda_nvcc cuda_llvm"
+hstack_qsv_filter_deps="libmfx"
+hstack_qsv_filter_select="qsvvpp"
+vstack_qsv_filter_deps="libmfx"
+vstack_qsv_filter_select="qsvvpp"
+xstack_qsv_filter_deps="libmfx"
+xstack_qsv_filter_select="qsvvpp"
 
 # examples
 avio_list_dir_deps="avformat avutil"
diff --git a/doc/filters.texi b/doc/filters.texi
index 790d165433..5d656a9351 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -13487,6 +13487,7 @@ Set the scaling dimension: @code{2} for @code{hq2x}, 
@code{3} for
 Default is @code{3}.
 @end table
 
+@anchor{hstack}
 @section hstack
 Stack input videos horizontally.
 
@@ -21861,6 +21862,7 @@ Example:
 ffmpeg -i ref.mpg -vf vmafmotion -f null -
 @end example
 
+@anchor{vstack}
 @section vstack
 Stack input videos vertically.
 
@@ -22307,6 +22309,7 @@ minimum values, and @code{1} maximum values.
 
 This filter supports all above options as @ref{commands}, excluding option 
@code{inputs}.
 
+@anchor{xstack}
 @section xstack
 Stack video inputs into custom layout.
 
@@ -24004,6 +24007,84 @@ tonemap_vaapi=format=p010:t=bt2020-10
 
 @c man end VAAPI VIDEO FILTERS
 
+@chapter QSV Video Filters
+@c man begin QSV VIDEO FILTERS
+
+Below is a description of the currently available QSV video filters.
+
+To enable compilation of these filters you need to configure FFmpeg with
+@code{--enable-libmfx}.
+
+To use QSV filters, you need to setup the QSV device correctly. For more 
information, please read @url{https://trac.ffmpeg.org/wiki/Hardware/QuickSync}
+
+@section hstack_qsv
+Stack input videos horizontally.
+
+This is the QSV variant of the @ref{hstack} filter.
+
+It accepts the following options:
+
+@table @option
+@item inputs
+Set number of input streams. Allowed range is from 2 to 72.
+Default value is 2.
+
+@item shortest
+See @ref{hstack}.
+
+@item scale
+Set factor to zoom in or out all videos. Allowed range is from 0.125 to 8.
+Default value is 1.
+@end table
+
+@section vstack_qsv
+Stack input videos vertically.
+
+This is the QSV variant of the @ref{vstack} filter.
+
+It accepts the following options:
+
+@table @option
+@item inputs
+Set number of input streams. Allowed range is from 2 to 72.
+Default value is 2.
+
+@item shortest
+See @ref{vstack}.
+
+@item scale
+Set factor to zoom in or out all videos. Allowed range is from 0.125 to 8.
+Default value is 1.
+@end table
+
+@section xstack_qsv
+Stack video inputs into custom layout.
+
+This is the QSV variant of the @ref{xstack} filter.
+
+It accepts the following options:
+
+@table @option
+@item inputs
+Set number of input streams. Allowed range is from 2 to 72.
+Default value is 2.
+
+@item shortest
+See @ref{xstack}.
+
+@item scale
+Set factor to zoom in or out all videos. Allowed range is from 0.125 to 8.
+Default value is 1.
+
+@item layout
+See @ref{xstack}.
+
+@item fill
+See @ref{xstack}.
+@end table
+
+@c man end QSV VIDEO FILTERS
+
 @chapter Video Sources
 @c man begin VIDEO SOURCES
 
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 49c0c8342b..03e8558635 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -496,6 +496,9 @@ OBJS-$(CONFIG_YAEPBLUR_FILTER)   += 
vf_yaepblur.o
 OBJS-$(CONFIG_ZMQ_FILTER)+= f_zmq.o
 OBJS-$(CONFIG_ZOOMPAN_FILTER)+= vf_zoompan.o
 OBJS-$(CONFIG_ZSCALE_FILTER) += vf_zscale.o
+OBJS-$(CONFIG_HSTACK_QSV_FILTER) += vf_stack_qsv.o framesync.o
+OBJS-$(CONFIG_VSTACK_QSV_FILTER) += vf_stack_qsv.o framesync.o
+OBJS-$(CONFIG_XSTACK_QSV_FILTER) += vf_stack_qsv.o framesync.o
 
 OBJS-$(CONFIG_ALLRGB_FILTER) += vsrc_testsrc.o
 OBJS-$(CONFIG_ALLYUV_FILTER) += vsrc_testsrc.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index ae74f9c891..16bc0c6429 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -472,6 +472,9 @@ extern const AVFilter ff_vf_yaepblur;
 extern const AVFilter ff_vf_zmq;
 extern const AVFilter ff_vf_zoompan;
 extern const AVFilter ff_vf_zscale;
+extern const AVFilter ff_vf_hstack_qsv;
+extern const AVFilter ff_vf_vstack_qsv;
+extern const AVFilter ff_vf_xstack_qsv;
 
 extern const AVFilter

Re: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the stack filters

2021-08-05 Thread Xiang, Haihao 
On Thu, 2021-08-05 at 15:53 +, Soft Works wrote:
> > -Original Message-
> > From: ffmpeg-devel  On Behalf Of
> > Xiang, Haihao
> > Sent: Thursday, 5 August 2021 04:33
> > To: ffmpeg-devel@ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the
> > stack filters
> > 
> > On Wed, 2021-08-04 at 09:17 +, Soft Works wrote:
> > > > -Original Message-
> > > > From: ffmpeg-devel  On Behalf Of
> > > > Haihao Xiang
> > > > Sent: Wednesday, 4 August 2021 10:33
> > > > To: ffmpeg-devel@ffmpeg.org
> > > > Cc: Haihao Xiang 
> > > > Subject: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the
> > > > stack filters
> > > > 
> > > > Include hstack_qsv, vstack_qsv and xstack_qsv, some code is copy and
> > > > pasted from other filters
> > > > 
> > > > Example:
> > > > $> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -filter_complex
> > > > "[0:v][0:v]hstack_qsv" -f null -
> > > > ---
> > > 
> > > [...]
> > > 
> > > > +
> > > > +/*
> > > > + * Callback for qsvvpp
> > > > + * @Note: qsvvpp composition does not generate PTS for result frame.
> > > > + *so we assign the PTS from framesync to the output frame.
> > > > + */
> > > > +
> > > > +static int filter_callback(AVFilterLink *outlink, AVFrame *frame) {
> > > > +QSVStackContext *sctx = outlink->src->priv;
> > > > +
> > > > +frame->pts = av_rescale_q(sctx->fs.pts,
> > > > +  sctx->fs.time_base, outlink->time_base);
> > > > +return ff_filter_frame(outlink, frame); }
> > > 
> > > If the surface.Data.TimeStamp gets overwritten by libMFX, why not copy
> > > the PTS from the input frame in ff_qsvvpp_filter_frame ?
> > > 
> > > That would apply the timestamp from the last input, though. Preferably
> > > would it be taken from the first input instead. For 2-n, you could
> > > perhaps clone the frames and assign the pts from the first input's
> > > frame?
> > 
> > Thanks for the comment and suggestion. This callback function was copy-
> > and- pasted from overlay_qsv filter because MSDK composition is also used
> > by this filter, I'd like to use the same way to generate pts for these
> > filters, but
> > I'll try your suggestion for these filters in the future.
> 
> Yea I see - the overlay_qsv filter does it the same way. This has probably
> been
> ok earlier because the callback happened synchronously. This is no longer the
> case since the async_depth patch which introduced the fifo processing. Now it
> can happen that the calback is performed for an earlier frame than the one
> that is currently gated by framesync.

async_depth is not enabled for overlay_qsv and stack qsv filters, 
s->async_depth 
is 0, so the callback is still performed synchronously for these filters. 

Thanks
Haihao

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the stack filters

2021-08-05 Thread Soft Works 



> -Original Message-
> From: ffmpeg-devel  On Behalf Of
> Xiang, Haihao
> Sent: Thursday, 5 August 2021 04:33
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the
> stack filters
> 
> On Wed, 2021-08-04 at 09:17 +, Soft Works wrote:
> > > -Original Message-
> > > From: ffmpeg-devel  On Behalf Of
> > > Haihao Xiang
> > > Sent: Wednesday, 4 August 2021 10:33
> > > To: ffmpeg-devel@ffmpeg.org
> > > Cc: Haihao Xiang 
> > > Subject: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the
> > > stack filters
> > >
> > > Include hstack_qsv, vstack_qsv and xstack_qsv, some code is copy and
> > > pasted from other filters
> > >
> > > Example:
> > > $> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -filter_complex
> > > "[0:v][0:v]hstack_qsv" -f null -
> > > ---
> >
> > [...]
> >
> > > +
> > > +/*
> > > + * Callback for qsvvpp
> > > + * @Note: qsvvpp composition does not generate PTS for result frame.
> > > + *so we assign the PTS from framesync to the output frame.
> > > + */
> > > +
> > > +static int filter_callback(AVFilterLink *outlink, AVFrame *frame) {
> > > +QSVStackContext *sctx = outlink->src->priv;
> > > +
> > > +frame->pts = av_rescale_q(sctx->fs.pts,
> > > +  sctx->fs.time_base, outlink->time_base);
> > > +return ff_filter_frame(outlink, frame); }
> >
> > If the surface.Data.TimeStamp gets overwritten by libMFX, why not copy
> > the PTS from the input frame in ff_qsvvpp_filter_frame ?
> >
> > That would apply the timestamp from the last input, though. Preferably
> > would it be taken from the first input instead. For 2-n, you could
> > perhaps clone the frames and assign the pts from the first input's
> > frame?
> 
> Thanks for the comment and suggestion. This callback function was copy-
> and- pasted from overlay_qsv filter because MSDK composition is also used
> by this filter, I'd like to use the same way to generate pts for these 
> filters, but
> I'll try your suggestion for these filters in the future.

Yea I see - the overlay_qsv filter does it the same way. This has probably been
ok earlier because the callback happened synchronously. This is no longer the
case since the async_depth patch which introduced the fifo processing. Now it
can happen that the calback is performed for an earlier frame than the one
that is currently gated by framesync.

I agree that this should be addressed in another patch.

Thanks,
sw
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the stack filters

2021-08-04 Thread Xiang, Haihao 
On Wed, 2021-08-04 at 09:17 +, Soft Works wrote:
> > -Original Message-
> > From: ffmpeg-devel  On Behalf Of
> > Haihao Xiang
> > Sent: Wednesday, 4 August 2021 10:33
> > To: ffmpeg-devel@ffmpeg.org
> > Cc: Haihao Xiang 
> > Subject: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the stack
> > filters
> > 
> > Include hstack_qsv, vstack_qsv and xstack_qsv, some code is copy and
> > pasted from other filters
> > 
> > Example:
> > $> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -filter_complex
> > "[0:v][0:v]hstack_qsv" -f null -
> > ---
> 
> [...]
> 
> > +
> > +/*
> > + * Callback for qsvvpp
> > + * @Note: qsvvpp composition does not generate PTS for result frame.
> > + *so we assign the PTS from framesync to the output frame.
> > + */
> > +
> > +static int filter_callback(AVFilterLink *outlink, AVFrame *frame)
> > +{
> > +QSVStackContext *sctx = outlink->src->priv;
> > +
> > +frame->pts = av_rescale_q(sctx->fs.pts,
> > +  sctx->fs.time_base, outlink->time_base);
> > +return ff_filter_frame(outlink, frame);
> > +}
> 
> If the surface.Data.TimeStamp gets overwritten by libMFX, why not copy 
> the PTS from the input frame in ff_qsvvpp_filter_frame ?
> 
> That would apply the timestamp from the last input, though. Preferably would
> it
> be taken from the first input instead. For 2-n, you could perhaps clone the
> frames and 
> assign the pts from the first input's frame?

Thanks for the comment and suggestion. This callback function was copy-and-
pasted from overlay_qsv filter because MSDK composition is also used by this
filter, I'd like to use the same way to generate pts for these filters, but I'll
try your suggestion for these filters in the future. 

Regards
Haihao

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the stack filters

2021-08-04 Thread Soft Works 



> -Original Message-
> From: ffmpeg-devel  On Behalf Of
> Haihao Xiang
> Sent: Wednesday, 4 August 2021 10:33
> To: ffmpeg-devel@ffmpeg.org
> Cc: Haihao Xiang 
> Subject: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the stack
> filters
> 
> Include hstack_qsv, vstack_qsv and xstack_qsv, some code is copy and
> pasted from other filters
> 
> Example:
> $> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -filter_complex
> "[0:v][0:v]hstack_qsv" -f null -
> ---

[...]

> +
> +/*
> + * Callback for qsvvpp
> + * @Note: qsvvpp composition does not generate PTS for result frame.
> + *so we assign the PTS from framesync to the output frame.
> + */
> +
> +static int filter_callback(AVFilterLink *outlink, AVFrame *frame)
> +{
> +QSVStackContext *sctx = outlink->src->priv;
> +
> +frame->pts = av_rescale_q(sctx->fs.pts,
> +  sctx->fs.time_base, outlink->time_base);
> +return ff_filter_frame(outlink, frame);
> +}

If the surface.Data.TimeStamp gets overwritten by libMFX, why not copy 
the PTS from the input frame in ff_qsvvpp_filter_frame ?

That would apply the timestamp from the last input, though. Preferably would it
be taken from the first input instead. For 2-n, you could perhaps clone the 
frames and 
assign the pts from the first input's frame?

softworkz

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the stack filters

2021-08-04 Thread Haihao Xiang 
Include hstack_qsv, vstack_qsv and xstack_qsv, some code is copy and
pasted from other filters

Example:
$> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -filter_complex
"[0:v][0:v]hstack_qsv" -f null -
---
v2: do not include avfilter.h and add docs for these filters

 configure  |   6 +
 doc/filters.texi   |  78 ++
 libavfilter/Makefile   |   3 +
 libavfilter/allfilters.c   |   3 +
 libavfilter/vf_stack_qsv.c | 498 +
 5 files changed, 588 insertions(+)
 create mode 100644 libavfilter/vf_stack_qsv.c

diff --git a/configure b/configure
index 94b30afe74..d29ae36006 100755
--- a/configure
+++ b/configure
@@ -3700,6 +3700,12 @@ vpp_qsv_filter_select="qsvvpp"
 xfade_opencl_filter_deps="opencl"
 yadif_cuda_filter_deps="ffnvcodec"
 yadif_cuda_filter_deps_any="cuda_nvcc cuda_llvm"
+hstack_qsv_filter_deps="libmfx"
+hstack_qsv_filter_select="qsvvpp"
+vstack_qsv_filter_deps="libmfx"
+vstack_qsv_filter_select="qsvvpp"
+xstack_qsv_filter_deps="libmfx"
+xstack_qsv_filter_select="qsvvpp"
 
 # examples
 avio_list_dir_deps="avformat avutil"
diff --git a/doc/filters.texi b/doc/filters.texi
index 66c0f87e47..0fdefabe96 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -23998,6 +23998,84 @@ tonemap_vaapi=format=p010:t=bt2020-10
 
 @c man end VAAPI VIDEO FILTERS
 
+@chapter QSV Video Filters
+@c man begin QSV VIDEO FILTERS
+
+Below is a description of the currently available QSV video filters.
+
+To enable compilation of these filters you need to configure FFmpeg with
+@code{--enable-libmfx}.
+
+To use QSV filters, you need to setup the QSV device correctly. For more 
information, please read @url{https://trac.ffmpeg.org/wiki/Hardware/QuickSync}
+
+@section hstack_qsv
+Stack input videos horizontally.
+
+This is the QSV variant of the @ref{hstack} filter.
+
+It accepts the following options:
+
+@table @option
+@item inputs
+Set number of input streams. Allowed range is from 2 to 72.
+Default value is 2.
+
+@item shortest
+See @ref{hstack}.
+
+@item scale
+Set factor to zoom in or out all videos. Allowed range is from 0.125 to 8.
+Default value is 1.
+@end table
+
+@section vstack_qsv
+Stack input videos vertically.
+
+This is the QSV variant of the @ref{vstack} filter.
+
+It accepts the following options:
+
+@table @option
+@item inputs
+Set number of input streams. Allowed range is from 2 to 72.
+Default value is 2.
+
+@item shortest
+See @ref{vstack}.
+
+@item scale
+Set factor to zoom in or out all videos. Allowed range is from 0.125 to 8.
+Default value is 1.
+@end table
+
+@section xstack_qsv
+Stack video inputs into custom layout.
+
+This is the QSV variant of the @ref{xstack_qsv} filter.
+
+It accepts the following options:
+
+@table @option
+@item inputs
+Set number of input streams. Allowed range is from 2 to 72.
+Default value is 2.
+
+@item shortest
+See @ref{xstack}.
+
+@item scale
+Set factor to zoom in or out all videos. Allowed range is from 0.125 to 8.
+Default value is 1.
+
+@item layout
+See @ref{xstack}.
+
+@item fill
+See @ref{xstack}.
+@end table
+
+@c man end QSV VIDEO FILTERS
+
 @chapter Video Sources
 @c man begin VIDEO SOURCES
 
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 49c0c8342b..03e8558635 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -496,6 +496,9 @@ OBJS-$(CONFIG_YAEPBLUR_FILTER)   += 
vf_yaepblur.o
 OBJS-$(CONFIG_ZMQ_FILTER)+= f_zmq.o
 OBJS-$(CONFIG_ZOOMPAN_FILTER)+= vf_zoompan.o
 OBJS-$(CONFIG_ZSCALE_FILTER) += vf_zscale.o
+OBJS-$(CONFIG_HSTACK_QSV_FILTER) += vf_stack_qsv.o framesync.o
+OBJS-$(CONFIG_VSTACK_QSV_FILTER) += vf_stack_qsv.o framesync.o
+OBJS-$(CONFIG_XSTACK_QSV_FILTER) += vf_stack_qsv.o framesync.o
 
 OBJS-$(CONFIG_ALLRGB_FILTER) += vsrc_testsrc.o
 OBJS-$(CONFIG_ALLYUV_FILTER) += vsrc_testsrc.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index ae74f9c891..16bc0c6429 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -472,6 +472,9 @@ extern const AVFilter ff_vf_yaepblur;
 extern const AVFilter ff_vf_zmq;
 extern const AVFilter ff_vf_zoompan;
 extern const AVFilter ff_vf_zscale;
+extern const AVFilter ff_vf_hstack_qsv;
+extern const AVFilter ff_vf_vstack_qsv;
+extern const AVFilter ff_vf_xstack_qsv;
 
 extern const AVFilter ff_vsrc_allrgb;
 extern const AVFilter ff_vsrc_allyuv;
diff --git a/libavfilter/vf_stack_qsv.c b/libavfilter/vf_stack_qsv.c
new file mode 100644
index 00..3d39e3ec60
--- /dev/null
+++ b/libavfilter/vf_stack_qsv.c
@@ -0,0 +1,498 @@
+/*
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that