Re: [FFmpeg-devel] [PATCH 1/2] dnn: add layer pad which is equivalent to tf.pad
> -Original Message-
> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On Behalf Of
> Pedro Arthur
> Sent: Friday, July 26, 2019 10:36 PM
> To: FFmpeg development discussions and patches
> Subject: Re: [FFmpeg-devel] [PATCH 1/2] dnn: add layer pad which is equivalent
> to tf.pad
>
> Hi,
>
> Em seg, 1 de jul de 2019 às 05:10, Guo, Yejun
> escreveu:
> >
> > +// handle the first dimension
> > +before_paddings = params->paddings[0][0];
> > +after_paddings = params->paddings[0][1];
> > +for (int n = 0; n < before_paddings; n++) {
> > +float *dst = output + n * new_hwc_stride;
> > +if (params->mode == LPMP_CONSTANT) {
> > +for (int i = 0; i < new_hwc_stride; i++) {
> > +*dst = params->constant_values;
> I suppose it is missing a pointer increase in dst ptr?
> I found this same pattern in all loops below.
thanks, nice catch.
I just checked my unit test for LPMP_CONSTANT and found it happened to test the
fourth dimension which does not need the pointer increasing.
will send out a new patch together with unit test improved.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/2] dnn: add layer pad which is equivalent to tf.pad
Hi,
Em seg, 1 de jul de 2019 às 05:10, Guo, Yejun escreveu:
>
> the reason to add this layer first is that vf_sr uses it in its
> tensorflow model, and the next plan is to update the python script
> to convert tf.pad into native model.
>
> Signed-off-by: Guo, Yejun
> ---
> libavfilter/dnn/Makefile | 1 +
> libavfilter/dnn/dnn_backend_native_layer_pad.c | 211
> +
> libavfilter/dnn/dnn_backend_native_layer_pad.h | 40 +
> 3 files changed, 252 insertions(+)
> create mode 100644 libavfilter/dnn/dnn_backend_native_layer_pad.c
> create mode 100644 libavfilter/dnn/dnn_backend_native_layer_pad.h
>
> diff --git a/libavfilter/dnn/Makefile b/libavfilter/dnn/Makefile
> index 1d12ade..83938e5 100644
> --- a/libavfilter/dnn/Makefile
> +++ b/libavfilter/dnn/Makefile
> @@ -1,5 +1,6 @@
> OBJS-$(CONFIG_DNN) += dnn/dnn_interface.o
> OBJS-$(CONFIG_DNN) += dnn/dnn_backend_native.o
> +OBJS-$(CONFIG_DNN) +=
> dnn/dnn_backend_native_layer_pad.o
>
> DNN-OBJS-$(CONFIG_LIBTENSORFLOW) += dnn/dnn_backend_tf.o
>
> diff --git a/libavfilter/dnn/dnn_backend_native_layer_pad.c
> b/libavfilter/dnn/dnn_backend_native_layer_pad.c
> new file mode 100644
> index 000..aa12f7f
> --- /dev/null
> +++ b/libavfilter/dnn/dnn_backend_native_layer_pad.c
> @@ -0,0 +1,211 @@
> +/*
> + * Copyright (c) 2019 Guo Yejun
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301
> USA
> + */
> +
> +#include
> +#include "libavutil/avassert.h"
> +#include "dnn_backend_native_layer_pad.h"
> +
> +static int before_get_buddy(int given, int paddings, LayerPadModeParam mode)
> +{
> +if (mode == LPMP_SYMMETRIC) {
> +return (2 * paddings - 1 - given);
> +} else if (mode == LPMP_REFLECT) {
> +return (2 * paddings - given);
> +} else {
> +av_assert0(!"should not reach here");
> +return 0;
> +}
> +}
> +
> +static int after_get_buddy(int given, int border, LayerPadModeParam mode)
> +{
> +if (mode == LPMP_SYMMETRIC) {
> +int offset = given - border;
> +return (border - 1 - offset);
> +} else if (mode == LPMP_REFLECT) {
> +int offset = given - border;
> +return (border - 2 - offset);
> +} else {
> +av_assert0(!"should not reach here");
> +return 0;
> +}
> +}
> +
> +void dnn_execute_layer_pad(const float *input, float *output, const
> LayerPadParams *params, int number, int height, int width, int channel)
> +{
> +int32_t before_paddings;
> +int32_t after_paddings;
> +
> +// suppose format is
> +int new_number = number + params->paddings[0][0] +
> params->paddings[0][1];
> +int new_height = height + params->paddings[1][0] +
> params->paddings[1][1];
> +int new_width = width + params->paddings[2][0] + params->paddings[2][1];
> +int new_channel = channel + params->paddings[3][0] +
> params->paddings[3][1];
> +
> +int c_stride = channel;
> +int wc_stride = c_stride * width;
> +int hwc_stride = wc_stride * height;
> +
> +int new_c_stride = new_channel;
> +int new_wc_stride = new_c_stride * new_width;
> +int new_hwc_stride = new_wc_stride * new_height;
> +
> +// copy the original data
> +for (int n = 0; n < number; n++) {
> +for (int h = 0; h < height; h++) {
> +for (int w = 0; w < width; w++) {
> +const float *src = input + n * hwc_stride + h * wc_stride +
> w * c_stride;
> +float *dst = output + (n + params->paddings[0][0]) *
> new_hwc_stride
> ++ (h + params->paddings[1][0]) *
> new_wc_stride
> ++ (w + params->paddings[2][0]) *
> new_c_stride
> ++ params->paddings[3][0];
> +memcpy(dst, src, channel * sizeof(float));
> +}
> +}
> +}
> +
> +// handle the first dimension
> +before_paddings = params->paddings[0][0];
> +after_paddings = params->paddings[0][1];
> +for (int n = 0; n < before_paddings; n++) {
> +float *dst = output + n * new_hwc_stride;
> +if (params->mode == LPMP_CONSTANT) {
>
Re: [FFmpeg-devel] [PATCH 1/2] dnn: add layer pad which is equivalent to tf.pad
> > -Original Message- > > From: Guo, Yejun > > Sent: Monday, July 01, 2019 4:08 PM > > To: ffmpeg-devel@ffmpeg.org > > Cc: Guo, Yejun > > Subject: [PATCH 1/2] dnn: add layer pad which is equivalent to tf.pad > > > > the reason to add this layer first is that vf_sr uses it in its > > tensorflow model, and the next plan is to update the python script > > to convert tf.pad into native model. > > > > Signed-off-by: Guo, Yejun > > this patch set is based on a previous patch at > http://ffmpeg.org/pipermail/ffmpeg-devel/2019-June/245481.html > (libavfilter/dnn: move dnn files from libavfilter to libavfilter/dnn) hello, I've finished 10+ new patches locally to improve dnn module, including one new patch relative to pad. I'll resend this patch set, together with the new pad relative patch. I'll send more patches set by set, once the previous patch set is pushed, since the patches have dependency. Just in case you are interested in my new patches, I've uploaded to https://github.com/guoyejun/ffmpeg/tree/dnn0716. for your convenient, I also copy the oneline log here for each patch (from newer to older) with 3 patch sets. 50a3353 fate: add unit test for dnn depth_to_space layer af9e3ab dnn: separate depth_to_space layer from dnn_backend_native.c to a new file 41b97e4 fate: add unit test for dnn conv2d layer 4143485 dnn: separate conv2d layer from dnn_backend_native.c to a new file 870383e dnn: export operand info in python script and load in c code 650d576 dnn: change .model file format to put layer number at the end of file d029bf8 dnn: introduce dnn operand (in c code) to hold operand infos within network c9b9e1c doc/filters: update how to generate native model for derain filter 064aa45 convert_from_tensorflow.py: support conv2d with dilation 1c419a5 convert_from_tensorflow.py: add option to dump graph for visualization in tensorboard I'll continue to improve dnn module, such as: - update dnn interface since we have more operand info (names, types, dims) in native mode. - optimize native conv2d layer - add more native layers to support at least mobile net as a milestone - add a general filter which generates analysis result (in a general side data for AVFrame) with dnn network. - add a general filter which changes the content of AVFrame with dnn network. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/2] dnn: add layer pad which is equivalent to tf.pad
> -Original Message- > From: Guo, Yejun > Sent: Monday, July 01, 2019 4:08 PM > To: ffmpeg-devel@ffmpeg.org > Cc: Guo, Yejun > Subject: [PATCH 1/2] dnn: add layer pad which is equivalent to tf.pad > > the reason to add this layer first is that vf_sr uses it in its > tensorflow model, and the next plan is to update the python script > to convert tf.pad into native model. > > Signed-off-by: Guo, Yejun this patch set is based on a previous patch at http://ffmpeg.org/pipermail/ffmpeg-devel/2019-June/245481.html (libavfilter/dnn: move dnn files from libavfilter to libavfilter/dnn) > --- > libavfilter/dnn/Makefile | 1 + > libavfilter/dnn/dnn_backend_native_layer_pad.c | 211 > + > libavfilter/dnn/dnn_backend_native_layer_pad.h | 40 + > 3 files changed, 252 insertions(+) > create mode 100644 libavfilter/dnn/dnn_backend_native_layer_pad.c > create mode 100644 libavfilter/dnn/dnn_backend_native_layer_pad.h > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
