> -----Original Message----- > From: Paul B Mahol [mailto:one...@gmail.com] > Sent: Wednesday, October 16, 2019 7:30 PM > To: Guo, Yejun <yejun....@intel.com> > Cc: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 4/4] avfilter: add a generic filter for rgb > proccessing with dnn networks > > On 10/16/19, Guo, Yejun <yejun....@intel.com> wrote: > > > > > >> -----Original Message----- > >> From: Paul B Mahol [mailto:one...@gmail.com] > >> Sent: Wednesday, October 16, 2019 5:17 PM > >> To: FFmpeg development discussions and patches > <ffmpeg-devel@ffmpeg.org> > >> Cc: Guo, Yejun <yejun....@intel.com> > >> Subject: Re: [FFmpeg-devel] [PATCH 4/4] avfilter: add a generic filter for > >> rgb > >> proccessing with dnn networks > >> > >> There should be only one dnn_processing filter. Not one that does only > >> rgb packed formats. > > > > Got it, I'll change it to dnn_processing and firstly implement the rgb > > format. > > > > For another possible case that multiple AVFrame are queued in the filter, it > > means that the dnn network needs more than one AVFrame, could it be a > > separate filter? Or it must be also integrated into dnn_processing? Thanks. > > Same filter, unless it needs multiple input/output pads, than needs > different name.
got it, thanks. > > > > > Btw, for the rest 3 patches in this patch set, they can be reviewed, the > > comment for this patch does not impact those patches. Thanks. > > > >> > >> On 10/16/19, Guo, Yejun <yejun....@intel.com> wrote: > >> > This filter accepts all the dnn networks which do image processing > >> > on RGB-based format. Currently, frame with formats rgb24 and bgr24 > >> > are supported. Other formats such as gray and YUV can be supported > >> > in separated filters. The dnn network can accept RGB data in float32 > >> > or uint8 format. And the dnn network can change frame size. > >> > > >> > Let's take an example with the following python script. This script > >> > halves the value of the first channel of the pixel. > >> > import tensorflow as tf > >> > import numpy as np > >> > import scipy.misc > >> > in_img = scipy.misc.imread('in.bmp') > >> > in_img = in_img.astype(np.float32)/255.0 > >> > in_data = in_img[np.newaxis, :] > >> > filter_data = np.array([0.5, 0, 0, 0, 1., 0, 0, 0, > >> > 1.]).reshape(1,1,3,3).astype(np.float32) > >> > filter = tf.Variable(filter_data) > >> > x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in') > >> > y = tf.nn.conv2d(x, filter, strides=[1, 1, 1, 1], padding='VALID', > >> > name='dnn_out') > >> > sess=tf.Session() > >> > sess.run(tf.global_variables_initializer()) > >> > output = sess.run(y, feed_dict={x: in_data}) > >> > graph_def = tf.graph_util.convert_variables_to_constants(sess, > >> > sess.graph_def, ['dnn_out']) > >> > tf.train.write_graph(graph_def, '.', 'halve_first_channel.pb', > >> > as_text=False) > >> > output = output * 255.0 > >> > output = output.astype(np.uint8) > >> > scipy.misc.imsave("out.bmp", np.squeeze(output)) > >> > > >> > - generate halve_first_channel.pb with the above script > >> > - generate halve_first_channel.model with tools/python/convert.py > >> > - try with following commands > >> > ./ffmpeg -i input.jpg -vf > >> > > >> > dnn_rgb_processing=model=halve_first_channel.model:input=dnn_in:output=d > >> nn_out:fmt=rgb24:dnn_backend=native > >> > -y out.native.png > >> > ./ffmpeg -i input.jpg -vf > >> > > >> > dnn_rgb_processing=model=halve_first_channel.pb:input=dnn_in:output=dnn_ > >> out:fmt=rgb24:dnn_backend=tensorflow > >> > -y out.tf.png > >> > > >> > Signed-off-by: Guo, Yejun <yejun....@intel.com> > >> > --- > >> > configure | 1 + > >> > doc/filters.texi | 46 ++++++ > >> > libavfilter/Makefile | 2 + > >> > libavfilter/allfilters.c | 1 + > >> > libavfilter/dnn_filter_utils.c | 81 +++++++++++ > >> > libavfilter/dnn_filter_utils.h | 35 +++++ > >> > libavfilter/vf_dnn_rgb_processing.c | 276 > >> > ++++++++++++++++++++++++++++++++++++ > >> > 7 files changed, 442 insertions(+) > >> > create mode 100644 libavfilter/dnn_filter_utils.c > >> > create mode 100644 libavfilter/dnn_filter_utils.h > >> > create mode 100644 libavfilter/vf_dnn_rgb_processing.c > > > > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".