From: Wenbin Chen
Add xpu device support to libtorch backend.
To enable xpu support you need to add
"-Wl,--no-as-needed -lintel-ext-pt-gpu -Wl,--as-needed" to
"--extra-libs" when configure ffmpeg.
Signed-off-by: Wenbin Chen
---
libavfilter/dnn/dnn_backend_torch.cpp | 16 +++-
1
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
libavfilter/dnn/dnn_io_proc.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/libavfilter/dnn/dnn_io_proc.c b/libavfilter/dnn/dnn_io_proc.c
index e5d6edb301..d2ec9f63f5 100644
--- a/libavfilter/dnn/dnn_io_proc.c
+++
From: Wenbin Chen
Check bbox's height with frame's height rather than frame's width.
Signed-off-by: Wenbin Chen
---
libavfilter/dnn/dnn_backend_openvino.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libavfilter/dnn/dnn_backend_openvino.c
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
doc/filters.texi | 12 +++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index 18f0d1c5a7..bfa8ccec8b 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -12073,11 +12073,21 @@
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
doc/filters.texi | 12 +++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index 913365671d..20605e72b2 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -12069,11 +12069,21 @@
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
Changelog | 1 +
1 file changed, 1 insertion(+)
diff --git a/Changelog b/Changelog
index e3ca52430c..4af55ff537 100644
--- a/Changelog
+++ b/Changelog
@@ -35,6 +35,7 @@ version :
- AEA muxer
- ffmpeg CLI loopback decoders
- Support
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
Changelog | 1 +
1 file changed, 1 insertion(+)
diff --git a/Changelog b/Changelog
index e3ca52430c..d0c41887f3 100644
--- a/Changelog
+++ b/Changelog
@@ -35,6 +35,7 @@ version :
- AEA muxer
- ffmpeg CLI loopback decoders
- Support
From: Wenbin Chen
PyTorch is an open source machine learning framework that accelerates
the path from research prototyping to production deployment. Official
website: https://pytorch.org/. We call the C++ library of PyTorch as
LibTorch, the same below.
To build FFmpeg with LibTorch, please take
From: Wenbin Chen
PyTorch is an open source machine learning framework that accelerates
the path from research prototyping to production deployment. Official
website: https://pytorch.org/. We call the C++ library of PyTorch as
LibTorch, the same below.
To build FFmpeg with LibTorch, please take
From: Wenbin Chen
PyTorch is an open source machine learning framework that accelerates
the path from research prototyping to production deployment. Official
website: https://pytorch.org/. We call the C++ library of PyTorch as
LibTorch, the same below.
To build FFmpeg with LibTorch, please take
From: Wenbin Chen
PyTorch is an open source machine learning framework that accelerates
the path from research prototyping to production deployment. Official
websit: https://pytorch.org/. We call the C++ library of PyTorch as
LibTorch, the same below.
To build FFmpeg with LibTorch, please take
From: Wenbin Chen
PyTorch is an open source machine learning framework that accelerates
the path from research prototyping to production deployment. Official
websit: https://pytorch.org/. We call the C++ library of PyTorch as
LibTorch, the same below.
To build FFmpeg with LibTorch, please take
From: Wenbin Chen
PyTorch is an open source machine learning framework that accelerates
the path from research prototyping to production deployment. Official
websit: https://pytorch.org/. We call the C++ library of PyTorch as
LibTorch, the same below.
To build FFmpeg with LibTorch, please take
From: Wenbin Chen
Use class confidence instead of box_score to filt boxes, which is more
accurate. Class confidence is obtained by multiplying class probability
distribution and box_score.
Signed-off-by: Wenbin Chen
---
libavfilter/vf_dnn_detect.c | 6 +++---
1 file changed, 3 insertions(+),
From: Wenbin Chen
For detect and classify output, width and height make no sence, so
change width, height to dims to represent the shape of tensor. Use
layout and dims to get width, height and channel.
Signed-off-by: Wenbin Chen
---
libavfilter/dnn/dnn_backend_openvino.c | 80
From: Wenbin Chen
Now when using openvino backend, user doesn't need to set input/output
names in command line. Model ports will be automatically detected.
For example:
ffmpeg -i input.png -vf \
dnn_detect=dnn_backend=openvino:model=model.xml:input=image:\
output=detection_out -y output.png
From: Wenbin Chen
For this kind of model, we can directly use its output as final result
just like ssd model. The difference is that it splits output into two
tensors. [x_min, y_min, x_max, y_max, confidence] and [lable_id].
Model example refer to:
From: Wenbin Chen
Add dynamic outputs support. Some models don't have fixed output size.
Its size changes according to result. Now openvino can run these kinds of
models.
Signed-off-by: Wenbin Chen
---
libavfilter/dnn/dnn_backend_openvino.c | 134 +++--
1 file changed, 59
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
libavfilter/vf_dnn_detect.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libavfilter/vf_dnn_detect.c b/libavfilter/vf_dnn_detect.c
index 52d5c3d798..88865c8a8e 100644
--- a/libavfilter/vf_dnn_detect.c
+++
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
libavfilter/vf_dnn_detect.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/libavfilter/vf_dnn_detect.c b/libavfilter/vf_dnn_detect.c
index fcc64118b6..52d5c3d798 100644
--- a/libavfilter/vf_dnn_detect.c
+++ b/libavfilter/vf_dnn_detect.c
From: Wenbin Chen
Set used pointer to NULL in case it leaks the storage.
Signed-off-by: Wenbin Chen
---
libavfilter/vf_dnn_detect.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/libavfilter/vf_dnn_detect.c b/libavfilter/vf_dnn_detect.c
index 5668b8b017..3464af86c8 100644
---
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
libavfilter/vf_dnn_detect.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/libavfilter/vf_dnn_detect.c b/libavfilter/vf_dnn_detect.c
index b2e9b8d4c6..5668b8b017 100644
--- a/libavfilter/vf_dnn_detect.c
+++
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
libavfilter/vf_dnn_detect.c | 4
1 file changed, 4 insertions(+)
diff --git a/libavfilter/vf_dnn_detect.c b/libavfilter/vf_dnn_detect.c
index b82916ce6d..b2e9b8d4c6 100644
--- a/libavfilter/vf_dnn_detect.c
+++ b/libavfilter/vf_dnn_detect.c
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
libavfilter/vf_dnn_detect.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libavfilter/vf_dnn_detect.c b/libavfilter/vf_dnn_detect.c
index 7ac3bb0b58..b82916ce6d 100644
--- a/libavfilter/vf_dnn_detect.c
+++
From: Wenbin Chen
Add yolov3 support. The difference of yolov3 is that it has multiple
outputs in different scale to perform better on both large and small
object.
The model detail refer to:
https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/yolo-v3-tf
Signed-off-by:
From: Wenbin Chen
Add multiple output support to openvino backend. You can use '&' to
split different output when you set output name using command line.
Signed-off-by: Wenbin Chen
---
libavfilter/dnn/dnn_backend_common.c | 7 -
libavfilter/dnn/dnn_backend_openvino.c | 216
From: Wenbin Chen
The difference of yolov4 is that sigmoid function needed to be applied
on x, y coordinates. Also make it compatiple with NHWC output as the
yolov4 model from openvino model zoo has NHWC output layout.
Model refer to:
From: Wenbin Chen
Add input pad to get model input resolution. Detection models always
have fixed input size. And the output coordinators are based on the
input resolution, so we need to get input size to map coordinators to
our real output frames.
Signed-off-by: Wenbin Chen
---
From: Wenbin Chen
The difference of yolov4 is that sigmoid function needed to be applied
on x, y coordinates. Also make it compatiple with NHWC output as the
yolov4 model from openvino model zoo has NHWC output layout.
Model refer to:
From: Wenbin Chen
Add yolov3 support. The difference of yolov3 is that it has multiple
outputs in different scale to perform better on both large and small
object.
The model detail refer to:
https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/yolo-v3-tf
Signed-off-by:
From: Wenbin Chen
Add input pad to get model input resolution. Detection models always
have fixed input size. And the output coordinators are based on the
input resolution, so we need to get input size to map coordinators to
our real output frames.
Signed-off-by: Wenbin Chen
---
From: Wenbin Chen
Add multiple output support to openvino backend. You can use '&' to
split different output when you set output name using command line.
Signed-off-by: Wenbin Chen
---
libavfilter/dnn/dnn_backend_common.c | 7 -
libavfilter/dnn/dnn_backend_openvino.c | 216
From: Wenbin Chen
Add yolo support. Yolo model doesn't output final result. It outputs
candidate boxes, so we need post-process to remove overlap boxes to
get final results. Also, the box's coordinators relate to cell and
anchors, so we need these information to calculate boxes as well.
Model
From: Wenbin Chen
There are many kinds of detection DNN model and they have different
preprocess and postprocess methods. To support more models,
"model_type" option is added to help to choose preprocess and
postprocess function.
Signed-off-by: Wenbin Chen
---
libavfilter/vf_dnn_detect.c | 42
From: Wenbin Chen
We can directly get data ptr from tensor, so that extral memory
allocation can be removed.
Signed-off-by: Wenbin Chen
---
libavfilter/dnn/dnn_backend_openvino.c | 42 +-
1 file changed, 21 insertions(+), 21 deletions(-)
diff --git
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
libavfilter/dnn/dnn_backend_tf.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/libavfilter/dnn/dnn_backend_tf.c b/libavfilter/dnn/dnn_backend_tf.c
index b521de7fbe..25046b58d9 100644
---
From: Wenbin Chen
Dnn models has different data preprocess requirements. Scale and mean
parameters are added to preprocess input data.
Signed-off-by: Wenbin Chen
---
libavfilter/dnn/dnn_backend_openvino.c | 43 --
libavfilter/dnn/dnn_io_proc.c | 82
From: Wenbin Chen
Dnn models have different input layout (NCHW or NHWC), so a
"layout" option is added
Use openvino's API to do layout conversion for input data. Use swscale
to do layout conversion for output data as openvino doesn't have
similiar C API for output.
Signed-off-by: Wenbin Chen
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
libavfilter/dnn/dnn_backend_tf.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/libavfilter/dnn/dnn_backend_tf.c b/libavfilter/dnn/dnn_backend_tf.c
index b521de7fbe..e1e8cef0d2 100644
--- a/libavfilter/dnn/dnn_backend_tf.c
From: Wenbin Chen
Dnn models has different data preprocess requirements. Scale and mean
parameters are added to preprocess input data.
Signed-off-by: Wenbin Chen
---
libavfilter/dnn/dnn_backend_openvino.c | 43 --
libavfilter/dnn/dnn_io_proc.c | 82
From: Wenbin Chen
Dnn models have different input layout (NCHW or NHWC), so a
"layout" option is added
Use openvino's API to do layout conversion for input data. Use swscale
to do layout conversion for output data as openvino doesn't have
similiar C API for output.
Signed-off-by: Wenbin Chen
From: Wenbin Chen
If user doesn't set framerate when he creates a filter, the filter uses
default framerate {0, 1}. This causes error when setting timebase to
1/framerate. Now change it to pass inlink->time_base to outlink when
framerate is not set.
This patch fixes ticket: #10476 #10468
From: Wenbin Chen
OpenVINO API 2.0 was released in March 2022, which introduced new
features.
This commit implements current OpenVINO features with new 2.0 APIs. And
will add other features in API 2.0.
Please add installation path, which include openvino.pc, to
PKG_CONFIG_PATH mannually for new
From: Wenbin Chen
Fix a bug that queried bbox index may bigger than bbox's total number.
Signed-off-by: Wenbin Chen
---
libavfilter/vf_dnn_detect.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/libavfilter/vf_dnn_detect.c b/libavfilter/vf_dnn_detect.c
index
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
doc/encoders.texi | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/doc/encoders.texi b/doc/encoders.texi
index b02737b9df..d6dddc2bd5 100644
--- a/doc/encoders.texi
+++ b/doc/encoders.texi
@@ -3188,8 +3188,8 @@ recommended
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
doc/examples/qsv_transcode.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/doc/examples/qsv_transcode.c b/doc/examples/qsv_transcode.c
index 7ea3ef5674..48128b200c 100644
--- a/doc/examples/qsv_transcode.c
+++
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
doc/encoders.texi | 2 +-
libavcodec/qsvenc.c | 5 -
2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/doc/encoders.texi b/doc/encoders.texi
index b02737b9df..e9b34010ed 100644
--- a/doc/encoders.texi
+++ b/doc/encoders.texi
From: Wenbin Chen
According to
https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#configuration-change.
Before calling MFXVideoENCODE_Reset, The application needs to retrieve
any cached frames in the SDK encoder.
A loop is added before MFXVideoENCODE_Reset to retrieve
From: Wenbin Chen
When encode RGB frame, Intel driver convert RGB to YUV, so we cannot
set RGB colorspace to VPL/MSDK.
Signed-off-by: Wenbin Chen
---
libavcodec/qsvenc.c | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c
index
From: Wenbin Chen
When encode RGB frame, Intel driver convert RGB to YUV, so we cannot
set RGB colorspace to VPL/MSDK.
Signed-off-by: Wenbin Chen
---
libavcodec/qsvenc.c | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c
index
From: Wenbin Chen
According to
https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#configuration-change.
Before calling MFXVideoENCODE_Reset, The application needs to retrieve
any cached frames in the SDK encoder.
A loop is added before MFXVideoENCODE_Reset to retrieve
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
libavfilter/qsvvpp.c | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c
index e181e7b584..54e7284234 100644
--- a/libavfilter/qsvvpp.c
+++ b/libavfilter/qsvvpp.c
@@ -441,7 +441,10
From: Wenbin Chen
MSDK/VPL uses 420 chroma format as default to encode RGB, and this is
not a proper usage. Now enable 444 encoding for RGB input by default.
RGB is encoded using 444 chroma format when user doesn't specify the
profile or uses rext profile, otherwise, 420 is used.
Signed-off-by:
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
doc/encoders.texi | 5 +
libavcodec/qsvenc.c | 3 +++
libavcodec/qsvenc_av1.c | 1 +
3 files changed, 9 insertions(+)
diff --git a/doc/encoders.texi b/doc/encoders.texi
index 543b5e26a9..727f12a59d 100644
--- a/doc/encoders.texi
From: Wenbin Chen
Signed-off-by: Wenbin Chen
---
doc/encoders.texi | 5 +
libavcodec/qsvenc.c | 4
libavcodec/qsvenc_av1.c | 1 +
3 files changed, 10 insertions(+)
diff --git a/doc/encoders.texi b/doc/encoders.texi
index b8051cda3f..543b5e26a9 100644
--- a/doc/encoders.texi
From: Wenbin Chen
When process yuv420 frames, FFmpeg uses same alignment on Y/U/V
planes. VPL and MSDK use Y plane's pitch / 2 as U/V planes's
pitch, which makes U/V planes 16-bytes aligned. We need to set
a separate alignment to meet runtime's behaviour.
Now alignment is changed to 16 so that
From: Wenbin Chen
When process yuv420 frames, FFmpeg uses same alignment on Y/U/V
planes. VPL and MSDK use Y plane's pitch / 2 as U/V planes's
pitch, which makes U/V planes 16-bytes aligned. We need to set
a separate alignment to meet runtime's behaviour.
Now alignment is changed to 16 so that
From: Wenbin Chen
When process yuv420 frames, FFmpeg use same alignment on Y/U/V
planes. VPL and MSDK use Y plane's pitch / 2 as U/V planes's
pitch, which make U/V planes 16-bytes aligned. We need to set
a separate alignment to meet runtime's behaviour.
Now the commandline works fine:
ffmpeg -f
From: Wenbin Chen
adaptive_i and adaptive_b cannot work with MFX_GOP_STRICT,
so only enable MFX_GOP_STRICT when these features are disabled.
Signed-off-by: Wenbin Chen
---
libavcodec/qsvenc.c | 9 -
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/libavcodec/qsvenc.c
From: Wenbin Chen
b_strategy, p_strategy, adaptive_i and adaptive_b cannot work with
MFX_GOP_STRICT, so disable MFX_GOP_STRICT when these features are used.
Signed-off-by: Wenbin Chen
---
libavcodec/qsvenc.c | 10 +-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git
From: Wenbin Chen
Add skip_frame support to qsvenc. Use per-frame metadata
"qsv_skip_frame" to control it. skip_frame option defines the behavior
of qsv_skip_frame.
no_skip: Frame skipping is disabled.
insert_dummy: Encoder inserts into bitstream frame where all macroblocks
are encoded as
From: Wenbin Chen
Add qsv_transcode example which shows how to use qsv to do hardware
accelerated transcoding, also show how to dynamically set encoding
parameters.
examples:
Normal usage:
qsv_transcode input.mp4 h264_qsv output.mp4 "g 60"
Dynamic setting usage:
qsv_transcode input.mp4
From: Wenbin Chen
Add doc for av1_qsv.
Signed-off-by: Wenbin Chen
---
doc/encoders.texi | 32
1 file changed, 32 insertions(+)
diff --git a/doc/encoders.texi b/doc/encoders.texi
index 1a5216f8eb..53dd02fd28 100644
--- a/doc/encoders.texi
+++
From: Wenbin Chen
It is available only when libvpl is enabled. MSDK doesn't support av1
encoding.
sample command:
ffmpeg -f rawvideo -pix_fmt nv12 -s 1920x1080 -i input.yuv \
-c:v av1_qsv output.ivf
Signed-off-by: Wenbin Chen
Signed-off-by: Haihao Xiang
---
configure | 2 +
64 matches
Mail list logo
