Re: [FFmpeg-devel] [PATCH 2/6] libavfilter: Unify Execution Modes in DNN Filters

2021-08-21 Thread Shubhanshu Saxena 
On Sat, Aug 21, 2021 at 12:56 PM Guo, Yejun  wrote:

>
>
> > -Original Message-
> > From: ffmpeg-devel  On Behalf Of
> > Shubhanshu Saxena
> > Sent: Saturday, August 21, 2021 3:05 PM
> > To: FFmpeg development discussions and patches  > de...@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [PATCH 2/6] libavfilter: Unify Execution
> Modes
> > in DNN Filters
> >
> > On Sat, Aug 21, 2021 at 8:41 AM Guo, Yejun  wrote:
> >
> > >
> > >
> > > > -Original Message-
> > > > From: ffmpeg-devel  On Behalf Of
> > > > Shubhanshu Saxena
> > > > Sent: 2021年8月20日 22:21
> > > > To: ffmpeg-devel@ffmpeg.org
> > > > Cc: Shubhanshu Saxena 
> > > > Subject: [FFmpeg-devel] [PATCH 2/6] libavfilter: Unify Execution
> > > > Modes in DNN Filters
> > > >
> > > > This commit unifies the async and sync mode from the DNN filters'
> > > > perspective. As of this commit, the Native backend only supports
> > > > synchronous execution mode.
> > > >
> > > > Now the user can switch between async and sync mode by using the
> > 'async'
> > > > option in the backend_configs. The values can be 1 for async and 0
> > > > for
> > > sync
> > > > mode of execution.
> > > >
> > > > This commit affects the following filters:
> > > > 1. vf_dnn_classify
> > > > 2. vf_dnn_detect
> > > > 3. vf_dnn_processing
> > > > 4. vf_sr
> > > > 5. vf_derain
> > > >
> > > > Signed-off-by: Shubhanshu Saxena 
> > > > ---
> > > >  libavfilter/dnn/dnn_backend_common.c   |  2 +-
> > > >  libavfilter/dnn/dnn_backend_common.h   |  5 +-
> > > >  libavfilter/dnn/dnn_backend_native.c   | 59 +++-
> > > >  libavfilter/dnn/dnn_backend_native.h   |  6 ++
> > > >  libavfilter/dnn/dnn_backend_openvino.c | 94
> > > > ++ libavfilter/dnn/dnn_backend_openvino.h |
> 3
> > +-
> > > >  libavfilter/dnn/dnn_backend_tf.c   | 35 ++
> > > >  libavfilter/dnn/dnn_backend_tf.h   |  3 +-
> > > >  libavfilter/dnn/dnn_interface.c|  8 +--
> > > >  libavfilter/dnn_filter_common.c| 23 +--
> > > >  libavfilter/dnn_filter_common.h|  3 +-
> > > >  libavfilter/dnn_interface.h|  4 +-
> > > >  libavfilter/vf_derain.c|  7 ++
> > > >  libavfilter/vf_dnn_classify.c  |  4 +-
> > > >  libavfilter/vf_dnn_detect.c|  8 +--
> > > >  libavfilter/vf_dnn_processing.c|  8 +--
> > > >  libavfilter/vf_sr.c|  8 +++
> > > >  17 files changed, 140 insertions(+), 140 deletions(-)
> > > >
> > >
> > > https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=4638 caught a
> > > warning:
> > > CC  libavfilter/vf_dnn_detect.o
> > > src/libavfilter/vf_dnn_detect.c:499:12: warning: ‘dnn_detect_activate’
> > > defined but not used [-Wunused-function]  static int
> > > dnn_detect_activate(AVFilterContext *filter_ctx)
> > > ^~~
> > > CC  libavfilter/vf_dnn_processing.o
> > > src/libavfilter/vf_dnn_processing.c:413:12: warning: ‘activate’
> > > defined but not used [-Wunused-function]  static int
> > > activate(AVFilterContext *filter_ctx)
> > > ^~~~
> > >
> > > I know it is fixed by the next patch, and the reason to separate these
> > > patches is for better change tracking.
> > >
> > > So, we can add 'av_unused' for these unused functions in this patch.
> > >
> > > ___
> > > ffmpeg-devel mailing list
> > > ffmpeg-devel@ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > >
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
> > >
> >
> > If I understood correctly, we need to rename these unused activate
> > functions to av_unused in this patch and then in the next patch fix
> these as
> > already done. Please correct me if I am wrong.
>
> Just add the following change in this patch 2.
>
> $ git diff
> diff --git a/libavfilter/vf_dnn_detect.c b/libavfilter/vf_dnn_detect.c
> index 2666abfcdc..11de376753 100644
> --- a/libav

Re: [FFmpeg-devel] [PATCH 2/6] libavfilter: Unify Execution Modes in DNN Filters

2021-08-21 Thread Guo, Yejun 


> -Original Message-
> From: ffmpeg-devel  On Behalf Of
> Shubhanshu Saxena
> Sent: Saturday, August 21, 2021 3:05 PM
> To: FFmpeg development discussions and patches  de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH 2/6] libavfilter: Unify Execution Modes
> in DNN Filters
> 
> On Sat, Aug 21, 2021 at 8:41 AM Guo, Yejun  wrote:
> 
> >
> >
> > > -Original Message-
> > > From: ffmpeg-devel  On Behalf Of
> > > Shubhanshu Saxena
> > > Sent: 2021年8月20日 22:21
> > > To: ffmpeg-devel@ffmpeg.org
> > > Cc: Shubhanshu Saxena 
> > > Subject: [FFmpeg-devel] [PATCH 2/6] libavfilter: Unify Execution
> > > Modes in DNN Filters
> > >
> > > This commit unifies the async and sync mode from the DNN filters'
> > > perspective. As of this commit, the Native backend only supports
> > > synchronous execution mode.
> > >
> > > Now the user can switch between async and sync mode by using the
> 'async'
> > > option in the backend_configs. The values can be 1 for async and 0
> > > for
> > sync
> > > mode of execution.
> > >
> > > This commit affects the following filters:
> > > 1. vf_dnn_classify
> > > 2. vf_dnn_detect
> > > 3. vf_dnn_processing
> > > 4. vf_sr
> > > 5. vf_derain
> > >
> > > Signed-off-by: Shubhanshu Saxena 
> > > ---
> > >  libavfilter/dnn/dnn_backend_common.c   |  2 +-
> > >  libavfilter/dnn/dnn_backend_common.h   |  5 +-
> > >  libavfilter/dnn/dnn_backend_native.c   | 59 +++-
> > >  libavfilter/dnn/dnn_backend_native.h   |  6 ++
> > >  libavfilter/dnn/dnn_backend_openvino.c | 94
> > > ++ libavfilter/dnn/dnn_backend_openvino.h |  3
> +-
> > >  libavfilter/dnn/dnn_backend_tf.c   | 35 ++
> > >  libavfilter/dnn/dnn_backend_tf.h   |  3 +-
> > >  libavfilter/dnn/dnn_interface.c|  8 +--
> > >  libavfilter/dnn_filter_common.c| 23 +--
> > >  libavfilter/dnn_filter_common.h|  3 +-
> > >  libavfilter/dnn_interface.h|  4 +-
> > >  libavfilter/vf_derain.c|  7 ++
> > >  libavfilter/vf_dnn_classify.c  |  4 +-
> > >  libavfilter/vf_dnn_detect.c|  8 +--
> > >  libavfilter/vf_dnn_processing.c|  8 +--
> > >  libavfilter/vf_sr.c|  8 +++
> > >  17 files changed, 140 insertions(+), 140 deletions(-)
> > >
> >
> > https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=4638 caught a
> > warning:
> > CC  libavfilter/vf_dnn_detect.o
> > src/libavfilter/vf_dnn_detect.c:499:12: warning: ‘dnn_detect_activate’
> > defined but not used [-Wunused-function]  static int
> > dnn_detect_activate(AVFilterContext *filter_ctx)
> > ^~~
> > CC  libavfilter/vf_dnn_processing.o
> > src/libavfilter/vf_dnn_processing.c:413:12: warning: ‘activate’
> > defined but not used [-Wunused-function]  static int
> > activate(AVFilterContext *filter_ctx)
> > ^~~~
> >
> > I know it is fixed by the next patch, and the reason to separate these
> > patches is for better change tracking.
> >
> > So, we can add 'av_unused' for these unused functions in this patch.
> >
> > ___
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
> >
> 
> If I understood correctly, we need to rename these unused activate
> functions to av_unused in this patch and then in the next patch fix these as
> already done. Please correct me if I am wrong.

Just add the following change in this patch 2.

$ git diff
diff --git a/libavfilter/vf_dnn_detect.c b/libavfilter/vf_dnn_detect.c
index 2666abfcdc..11de376753 100644
--- a/libavfilter/vf_dnn_detect.c
+++ b/libavfilter/vf_dnn_detect.c
@@ -496,7 +496,7 @@ static int dnn_detect_activate_async(AVFilterContext 
*filter_ctx)
 return 0;
 }

-static int dnn_detect_activate(AVFilterContext *filter_ctx)
+static av_unused int dnn_detect_activate(AVFilterContext *filter_ctx)
 {
 DnnDetectContext *ctx = filter_ctx->priv;

diff --git a/libavfilter/vf_dnn_processing.c b/libavfilter/vf_dnn_processing.c
index 410cc887dc..7435dd4959 100644
--- a/libavfilter/vf_dnn_processing.c
+++ b/libavfilter/vf_dnn_processing.c
@@ -410,7 +410,7 @@ static int activate_async(AVFilterContext *filter_ctx)
 return 0;
 }

-static int activate(AVFilterContext *filter_ctx)
+static av_unused int activate(AVFilterContext *filter_ctx)
 {
 DnnProcessingContext *ctx = filter_ctx->priv;

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/6] libavfilter: Unify Execution Modes in DNN Filters

2021-08-21 Thread Shubhanshu Saxena 
On Sat, Aug 21, 2021 at 8:41 AM Guo, Yejun  wrote:

>
>
> > -Original Message-
> > From: ffmpeg-devel  On Behalf Of
> > Shubhanshu Saxena
> > Sent: 2021年8月20日 22:21
> > To: ffmpeg-devel@ffmpeg.org
> > Cc: Shubhanshu Saxena 
> > Subject: [FFmpeg-devel] [PATCH 2/6] libavfilter: Unify Execution Modes in
> > DNN Filters
> >
> > This commit unifies the async and sync mode from the DNN filters'
> > perspective. As of this commit, the Native backend only supports
> > synchronous execution mode.
> >
> > Now the user can switch between async and sync mode by using the 'async'
> > option in the backend_configs. The values can be 1 for async and 0 for
> sync
> > mode of execution.
> >
> > This commit affects the following filters:
> > 1. vf_dnn_classify
> > 2. vf_dnn_detect
> > 3. vf_dnn_processing
> > 4. vf_sr
> > 5. vf_derain
> >
> > Signed-off-by: Shubhanshu Saxena 
> > ---
> >  libavfilter/dnn/dnn_backend_common.c   |  2 +-
> >  libavfilter/dnn/dnn_backend_common.h   |  5 +-
> >  libavfilter/dnn/dnn_backend_native.c   | 59 +++-
> >  libavfilter/dnn/dnn_backend_native.h   |  6 ++
> >  libavfilter/dnn/dnn_backend_openvino.c | 94 ++
> > libavfilter/dnn/dnn_backend_openvino.h |  3 +-
> >  libavfilter/dnn/dnn_backend_tf.c   | 35 ++
> >  libavfilter/dnn/dnn_backend_tf.h   |  3 +-
> >  libavfilter/dnn/dnn_interface.c|  8 +--
> >  libavfilter/dnn_filter_common.c| 23 +--
> >  libavfilter/dnn_filter_common.h|  3 +-
> >  libavfilter/dnn_interface.h|  4 +-
> >  libavfilter/vf_derain.c|  7 ++
> >  libavfilter/vf_dnn_classify.c  |  4 +-
> >  libavfilter/vf_dnn_detect.c|  8 +--
> >  libavfilter/vf_dnn_processing.c|  8 +--
> >  libavfilter/vf_sr.c|  8 +++
> >  17 files changed, 140 insertions(+), 140 deletions(-)
> >
>
> https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=4638 caught a
> warning:
> CC  libavfilter/vf_dnn_detect.o
> src/libavfilter/vf_dnn_detect.c:499:12: warning: ‘dnn_detect_activate’
> defined but not used [-Wunused-function]
>  static int dnn_detect_activate(AVFilterContext *filter_ctx)
> ^~~
> CC  libavfilter/vf_dnn_processing.o
> src/libavfilter/vf_dnn_processing.c:413:12: warning: ‘activate’ defined
> but not used [-Wunused-function]
>  static int activate(AVFilterContext *filter_ctx)
> ^~~~
>
> I know it is fixed by the next patch, and the reason to separate these
> patches is for better change tracking.
>
> So, we can add 'av_unused' for these unused functions in this patch.
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
>

If I understood correctly, we need to rename these unused activate
functions to av_unused in this patch and then in the next patch fix these
as already done. Please correct me if I am wrong.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/6] libavfilter: Unify Execution Modes in DNN Filters

2021-08-20 Thread Guo, Yejun 


> -Original Message-
> From: ffmpeg-devel  On Behalf Of
> Shubhanshu Saxena
> Sent: 2021年8月20日 22:21
> To: ffmpeg-devel@ffmpeg.org
> Cc: Shubhanshu Saxena 
> Subject: [FFmpeg-devel] [PATCH 2/6] libavfilter: Unify Execution Modes in
> DNN Filters
> 
> This commit unifies the async and sync mode from the DNN filters'
> perspective. As of this commit, the Native backend only supports
> synchronous execution mode.
> 
> Now the user can switch between async and sync mode by using the 'async'
> option in the backend_configs. The values can be 1 for async and 0 for sync
> mode of execution.
> 
> This commit affects the following filters:
> 1. vf_dnn_classify
> 2. vf_dnn_detect
> 3. vf_dnn_processing
> 4. vf_sr
> 5. vf_derain
> 
> Signed-off-by: Shubhanshu Saxena 
> ---
>  libavfilter/dnn/dnn_backend_common.c   |  2 +-
>  libavfilter/dnn/dnn_backend_common.h   |  5 +-
>  libavfilter/dnn/dnn_backend_native.c   | 59 +++-
>  libavfilter/dnn/dnn_backend_native.h   |  6 ++
>  libavfilter/dnn/dnn_backend_openvino.c | 94 ++
> libavfilter/dnn/dnn_backend_openvino.h |  3 +-
>  libavfilter/dnn/dnn_backend_tf.c   | 35 ++
>  libavfilter/dnn/dnn_backend_tf.h   |  3 +-
>  libavfilter/dnn/dnn_interface.c|  8 +--
>  libavfilter/dnn_filter_common.c| 23 +--
>  libavfilter/dnn_filter_common.h|  3 +-
>  libavfilter/dnn_interface.h|  4 +-
>  libavfilter/vf_derain.c|  7 ++
>  libavfilter/vf_dnn_classify.c  |  4 +-
>  libavfilter/vf_dnn_detect.c|  8 +--
>  libavfilter/vf_dnn_processing.c|  8 +--
>  libavfilter/vf_sr.c|  8 +++
>  17 files changed, 140 insertions(+), 140 deletions(-)
> 

https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=4638 caught a warning:
CC  libavfilter/vf_dnn_detect.o
src/libavfilter/vf_dnn_detect.c:499:12: warning: ‘dnn_detect_activate’ defined 
but not used [-Wunused-function]
 static int dnn_detect_activate(AVFilterContext *filter_ctx)
^~~
CC  libavfilter/vf_dnn_processing.o
src/libavfilter/vf_dnn_processing.c:413:12: warning: ‘activate’ defined but not 
used [-Wunused-function]
 static int activate(AVFilterContext *filter_ctx)
^~~~

I know it is fixed by the next patch, and the reason to separate these patches 
is for better change tracking.

So, we can add 'av_unused' for these unused functions in this patch.

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/6] libavfilter: Unify Execution Modes in DNN Filters

2021-08-20 Thread Shubhanshu Saxena 
This commit unifies the async and sync mode from the DNN filters'
perspective. As of this commit, the Native backend only supports
synchronous execution mode.

Now the user can switch between async and sync mode by using the
'async' option in the backend_configs. The values can be 1 for
async and 0 for sync mode of execution.

This commit affects the following filters:
1. vf_dnn_classify
2. vf_dnn_detect
3. vf_dnn_processing
4. vf_sr
5. vf_derain

Signed-off-by: Shubhanshu Saxena 
---
 libavfilter/dnn/dnn_backend_common.c   |  2 +-
 libavfilter/dnn/dnn_backend_common.h   |  5 +-
 libavfilter/dnn/dnn_backend_native.c   | 59 +++-
 libavfilter/dnn/dnn_backend_native.h   |  6 ++
 libavfilter/dnn/dnn_backend_openvino.c | 94 ++
 libavfilter/dnn/dnn_backend_openvino.h |  3 +-
 libavfilter/dnn/dnn_backend_tf.c   | 35 ++
 libavfilter/dnn/dnn_backend_tf.h   |  3 +-
 libavfilter/dnn/dnn_interface.c|  8 +--
 libavfilter/dnn_filter_common.c| 23 +--
 libavfilter/dnn_filter_common.h|  3 +-
 libavfilter/dnn_interface.h|  4 +-
 libavfilter/vf_derain.c|  7 ++
 libavfilter/vf_dnn_classify.c  |  4 +-
 libavfilter/vf_dnn_detect.c|  8 +--
 libavfilter/vf_dnn_processing.c|  8 +--
 libavfilter/vf_sr.c|  8 +++
 17 files changed, 140 insertions(+), 140 deletions(-)

diff --git a/libavfilter/dnn/dnn_backend_common.c 
b/libavfilter/dnn/dnn_backend_common.c
index 426683b73d..d2bc016fef 100644
--- a/libavfilter/dnn/dnn_backend_common.c
+++ b/libavfilter/dnn/dnn_backend_common.c
@@ -138,7 +138,7 @@ DNNReturnType ff_dnn_start_inference_async(void *ctx, 
DNNAsyncExecModule *async_
 return DNN_SUCCESS;
 }
 
-DNNAsyncStatusType ff_dnn_get_async_result_common(Queue *task_queue, AVFrame 
**in, AVFrame **out)
+DNNAsyncStatusType ff_dnn_get_result_common(Queue *task_queue, AVFrame **in, 
AVFrame **out)
 {
 TaskItem *task = ff_queue_peek_front(task_queue);
 
diff --git a/libavfilter/dnn/dnn_backend_common.h 
b/libavfilter/dnn/dnn_backend_common.h
index 604c1d3bd7..78e62a94a2 100644
--- a/libavfilter/dnn/dnn_backend_common.h
+++ b/libavfilter/dnn/dnn_backend_common.h
@@ -29,7 +29,8 @@
 #include "libavutil/thread.h"
 
 #define DNN_BACKEND_COMMON_OPTIONS \
-{ "nireq",   "number of request", 
OFFSET(options.nireq),   AV_OPT_TYPE_INT,{ .i64 = 0 }, 0, 
INT_MAX, FLAGS },
+{ "nireq",   "number of request", 
OFFSET(options.nireq),   AV_OPT_TYPE_INT,{ .i64 = 0 }, 0, 
INT_MAX, FLAGS }, \
+{ "async",   "use DNN async inference",   
OFFSET(options.async),   AV_OPT_TYPE_BOOL,   { .i64 = 1 }, 0,   
1, FLAGS },
 
 // one task for one function call from dnn interface
 typedef struct TaskItem {
@@ -135,7 +136,7 @@ DNNReturnType ff_dnn_start_inference_async(void *ctx, 
DNNAsyncExecModule *async_
  * @retval DAST_NOT_READY if inference not completed yet.
  * @retval DAST_SUCCESS if result successfully extracted
  */
-DNNAsyncStatusType ff_dnn_get_async_result_common(Queue *task_queue, AVFrame 
**in, AVFrame **out);
+DNNAsyncStatusType ff_dnn_get_result_common(Queue *task_queue, AVFrame **in, 
AVFrame **out);
 
 /**
  * Allocate input and output frames and fill the Task
diff --git a/libavfilter/dnn/dnn_backend_native.c 
b/libavfilter/dnn/dnn_backend_native.c
index 3b2a3aa55d..2d34b88f8a 100644
--- a/libavfilter/dnn/dnn_backend_native.c
+++ b/libavfilter/dnn/dnn_backend_native.c
@@ -34,6 +34,7 @@
 #define FLAGS AV_OPT_FLAG_FILTERING_PARAM
 static const AVOption dnn_native_options[] = {
 { "conv2d_threads", "threads num for conv2d layer", 
OFFSET(options.conv2d_threads), AV_OPT_TYPE_INT,  { .i64 = 0 }, INT_MIN, 
INT_MAX, FLAGS },
+{ "async",  "use DNN async inference",  OFFSET(options.async), 
 AV_OPT_TYPE_BOOL, { .i64 = 0 },   0,   1, FLAGS },
 { NULL },
 };
 
@@ -189,6 +190,11 @@ DNNModel *ff_dnn_load_model_native(const char 
*model_filename, DNNFunctionType f
 goto fail;
 native_model->model = model;
 
+if (native_model->ctx.options.async) {
+av_log(&native_model->ctx, AV_LOG_WARNING, "Async not supported. 
Rolling back to sync\n");
+native_model->ctx.options.async = 0;
+}
+
 #if !HAVE_PTHREAD_CANCEL
 if (native_model->ctx.options.conv2d_threads > 1){
 av_log(&native_model->ctx, AV_LOG_WARNING, "'conv2d_threads' option 
was set but it is not supported "
@@ -212,6 +218,11 @@ DNNModel *ff_dnn_load_model_native(const char 
*model_filename, DNNFunctionType f
 goto fail;
 }
 
+native_model->task_queue = ff_queue_create();
+if (!native_model->task_queue) {
+goto fail;
+}
+
 native_model->inference_queue = ff_queue_create();
 if (!native_model->inference_queue) {
 goto fail;
@@ -425,17 +436,30 @@ DNNReturnType ff_dnn_execute_model_native(const DNNModel 
*model, DNNExe