Re: [FFmpeg-devel] [GSOC] [PATCH] SRCNN filter

[James Almer]

On 5/7/2018 8:05 PM, Hendrik Leppkes wrote:
> On Sun, May 6, 2018 at 10:27 PM, James Almer  wrote:
>> On 5/5/2018 5:38 PM, James Almer wrote:
>>> On 4/10/2018 2:16 PM, Sergey Lavrushkin wrote:
 diff --git a/libavfilter/vf_srcnn.c b/libavfilter/vf_srcnn.c
 new file mode 100644
 index 00..d9b4891f7f
 --- /dev/null
 +++ b/libavfilter/vf_srcnn.c
 @@ -0,0 +1,420 @@
 +/*
 + * Copyright (c) 2018 Sergey Lavrushkin
 + *
 + * This file is part of FFmpeg.
 + *
 + * FFmpeg is free software; you can redistribute it and/or
 + * modify it under the terms of the GNU Lesser General Public
 + * License as published by the Free Software Foundation; either
 + * version 2.1 of the License, or (at your option) any later version.
 + *
 + * FFmpeg is distributed in the hope that it will be useful,
 + * but WITHOUT ANY WARRANTY; without even the implied warranty of
 + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 + * Lesser General Public License for more details.
 + *
 + * You should have received a copy of the GNU Lesser General Public
 + * License along with FFmpeg; if not, write to the Free Software
 + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 
 02110-1301 USA
 + */
 +
 +/**
 + * @file
 + * Filter implementing image super-resolution using deep convolutional 
 networks.
 + * https://arxiv.org/abs/1501.00092
 + */
 +
 +#include "avfilter.h"
 +#include "formats.h"
 +#include "internal.h"
 +#include "libavutil/opt.h"
 +#include "unistd.h"
>>>
>>> This broke msvc. It should be (if anything) , and wrapped
>>> inside a HAVE_UNISTD_H preprocessor check.
>>
>> I (accidentally) applied this change as part of an unrelated commit, so
>> that's dealt with at least.
>>
>>>
 +static av_cold int init(AVFilterContext* context)
 +{
 +SRCNNContext *srcnn_context = context->priv;
 +AVIOContext* config_file_context;
 +int64_t file_size, srcnn_size;
 +
 +/// Check specified confguration file name and read network weights 
 from it
 +if (!srcnn_context->config_file_path){
 +av_log(context, AV_LOG_INFO, "configuration file for network was 
 not specified, using default weights for x2 upsampling\n");
 +
 +/// Create convolution kernels and copy default weights
 +srcnn_context->conv1.input_channels = 1;
 +srcnn_context->conv1.output_channels = 64;
 +srcnn_context->conv1.size = 9;
 +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv1, 
 conv1_kernel, conv1_biases), )
 +
 +srcnn_context->conv2.input_channels = 64;
 +srcnn_context->conv2.output_channels = 32;
 +srcnn_context->conv2.size = 1;
 +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv2, 
 conv2_kernel, conv2_biases), )
 +
 +srcnn_context->conv3.input_channels = 32;
 +srcnn_context->conv3.output_channels = 1;
 +srcnn_context->conv3.size = 5;
 +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv3, 
 conv3_kernel, conv3_biases), )
 +}
 +else if (access(srcnn_context->config_file_path, R_OK) != -1){
>>>
>>> This seems to be the function needed from unistd.h
>>>
>>> Looking at configure and libavformat/file.c it's apparently not
>>> available everywhere (we have a HAVE_ACCESS define, and also check for
>>> mode constants like R_OK).
>>
>> MSVC does not have access(), so i have temporarily made this filter
>> depend on its presence until a proper fix is applied. That target has
>> been broken for a few days now and that shouldn't be the case.
>> It should be a matter or doing the same thing as in libavformat/file.c,
>> i guess, or perhaps use _access() instead, which is available in MSVC.
> 
> Your check doesn't work, MSVC building is still broken.
> Probably because MSVC does have access, in io.h, but it does not have
> the R_OK token.
> 
> - Hendrik

I sent a patch to make the access() check stricter.

access() in msvc is in any case kinda weird. The check as is succeeds
(so the function exists in some form in the c library) so HAVE_ACCESS is
set to true in both msvc and mingw, but msvc doesn't define R_OK or any
other such constant.
We have a wrapper using msvc's specific _access() in lavf that's
ultimately only used by mingw because the only file calling access() is
file.c and it first makes sure said constants are also set.

With my configure patch the check will only succeed on mingw, so
technically changing nothing as far as lavf is concerned, but
effectively disabling the filter on msvc so lavfi can compile.

Better solution of course welcome.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [GSOC] [PATCH] SRCNN filter

[Hendrik Leppkes]

On Sun, May 6, 2018 at 10:27 PM, James Almer  wrote:
> On 5/5/2018 5:38 PM, James Almer wrote:
>> On 4/10/2018 2:16 PM, Sergey Lavrushkin wrote:
>>> diff --git a/libavfilter/vf_srcnn.c b/libavfilter/vf_srcnn.c
>>> new file mode 100644
>>> index 00..d9b4891f7f
>>> --- /dev/null
>>> +++ b/libavfilter/vf_srcnn.c
>>> @@ -0,0 +1,420 @@
>>> +/*
>>> + * Copyright (c) 2018 Sergey Lavrushkin
>>> + *
>>> + * This file is part of FFmpeg.
>>> + *
>>> + * FFmpeg is free software; you can redistribute it and/or
>>> + * modify it under the terms of the GNU Lesser General Public
>>> + * License as published by the Free Software Foundation; either
>>> + * version 2.1 of the License, or (at your option) any later version.
>>> + *
>>> + * FFmpeg is distributed in the hope that it will be useful,
>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>>> + * Lesser General Public License for more details.
>>> + *
>>> + * You should have received a copy of the GNU Lesser General Public
>>> + * License along with FFmpeg; if not, write to the Free Software
>>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 
>>> 02110-1301 USA
>>> + */
>>> +
>>> +/**
>>> + * @file
>>> + * Filter implementing image super-resolution using deep convolutional 
>>> networks.
>>> + * https://arxiv.org/abs/1501.00092
>>> + */
>>> +
>>> +#include "avfilter.h"
>>> +#include "formats.h"
>>> +#include "internal.h"
>>> +#include "libavutil/opt.h"
>>> +#include "unistd.h"
>>
>> This broke msvc. It should be (if anything) , and wrapped
>> inside a HAVE_UNISTD_H preprocessor check.
>
> I (accidentally) applied this change as part of an unrelated commit, so
> that's dealt with at least.
>
>>
>>> +static av_cold int init(AVFilterContext* context)
>>> +{
>>> +SRCNNContext *srcnn_context = context->priv;
>>> +AVIOContext* config_file_context;
>>> +int64_t file_size, srcnn_size;
>>> +
>>> +/// Check specified confguration file name and read network weights 
>>> from it
>>> +if (!srcnn_context->config_file_path){
>>> +av_log(context, AV_LOG_INFO, "configuration file for network was 
>>> not specified, using default weights for x2 upsampling\n");
>>> +
>>> +/// Create convolution kernels and copy default weights
>>> +srcnn_context->conv1.input_channels = 1;
>>> +srcnn_context->conv1.output_channels = 64;
>>> +srcnn_context->conv1.size = 9;
>>> +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv1, 
>>> conv1_kernel, conv1_biases), )
>>> +
>>> +srcnn_context->conv2.input_channels = 64;
>>> +srcnn_context->conv2.output_channels = 32;
>>> +srcnn_context->conv2.size = 1;
>>> +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv2, 
>>> conv2_kernel, conv2_biases), )
>>> +
>>> +srcnn_context->conv3.input_channels = 32;
>>> +srcnn_context->conv3.output_channels = 1;
>>> +srcnn_context->conv3.size = 5;
>>> +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv3, 
>>> conv3_kernel, conv3_biases), )
>>> +}
>>> +else if (access(srcnn_context->config_file_path, R_OK) != -1){
>>
>> This seems to be the function needed from unistd.h
>>
>> Looking at configure and libavformat/file.c it's apparently not
>> available everywhere (we have a HAVE_ACCESS define, and also check for
>> mode constants like R_OK).
>
> MSVC does not have access(), so i have temporarily made this filter
> depend on its presence until a proper fix is applied. That target has
> been broken for a few days now and that shouldn't be the case.
> It should be a matter or doing the same thing as in libavformat/file.c,
> i guess, or perhaps use _access() instead, which is available in MSVC.

Your check doesn't work, MSVC building is still broken.
Probably because MSVC does have access, in io.h, but it does not have
the R_OK token.

- Hendrik
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [GSOC] [PATCH] SRCNN filter

[Sergey Lavrushkin]

2018-05-07 17:41 GMT+03:00 Pedro Arthur :

> 2018-05-07 0:30 GMT-03:00 Steven Liu :
> > Hi Sergey,
> >
> > How should i test this filter?
> > I tested it some days ago, the picture get worse from 2nd frame.
> > input resolution 640x480 to 1280x720;
> >
> > ffmpeg -i input -vf srcnn output
> Hi,
> The filter expects the input upscaled by 2x, therefore the proper
> command would be
>
> ffmpeg -i input -vf "scale=2*iw:2*ih, srcnn, scale=1280:720"
>
> The default filter is trained for 2x upscale, anything different from
> that may generate bad results.


Hi,
Moreover, the filter expects the input upscaled with bicubic upscaling, so
for other upscaling algorithms bad results are also possible.
Also other models for x2, x3, x4 upsampling can be specified using
following command:
ffmpeg -i input -vf scale=iw*factor:-1,srcnn=path_to_model
Configuration files with other models can be found here:
https://drive.google.com/drive/folders/1-M9azWTtZ4egf8ndRU7Y_tiGP6QtN-Fp?usp=sharing
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [GSOC] [PATCH] SRCNN filter

[Pedro Arthur]

2018-05-07 0:30 GMT-03:00 Steven Liu :
> Hi Sergey,
>
> How should i test this filter?
> I tested it some days ago, the picture get worse from 2nd frame.
> input resolution 640x480 to 1280x720;
>
> ffmpeg -i input -vf srcnn output
Hi,
The filter expects the input upscaled by 2x, therefore the proper
command would be

ffmpeg -i input -vf "scale=2*iw:2*ih, srcnn, scale=1280:720"

The default filter is trained for 2x upscale, anything different from
that may generate bad results.

>
>> On May 7, 2018, at 09:33, Pedro Arthur  wrote:
>>
>> 2018-05-06 17:27 GMT-03:00 James Almer :
>>> On 5/5/2018 5:38 PM, James Almer wrote:
 On 4/10/2018 2:16 PM, Sergey Lavrushkin wrote:
> diff --git a/libavfilter/vf_srcnn.c b/libavfilter/vf_srcnn.c
> new file mode 100644
> index 00..d9b4891f7f
> --- /dev/null
> +++ b/libavfilter/vf_srcnn.c
> @@ -0,0 +1,420 @@
> +/*
> + * Copyright (c) 2018 Sergey Lavrushkin
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 
> 02110-1301 USA
> + */
> +
> +/**
> + * @file
> + * Filter implementing image super-resolution using deep convolutional 
> networks.
> + * https://arxiv.org/abs/1501.00092
> + */
> +
> +#include "avfilter.h"
> +#include "formats.h"
> +#include "internal.h"
> +#include "libavutil/opt.h"
> +#include "unistd.h"

 This broke msvc. It should be (if anything) , and wrapped
 inside a HAVE_UNISTD_H preprocessor check.
>>>
>>> I (accidentally) applied this change as part of an unrelated commit, so
>>> that's dealt with at least.
>>>

> +static av_cold int init(AVFilterContext* context)
> +{
> +SRCNNContext *srcnn_context = context->priv;
> +AVIOContext* config_file_context;
> +int64_t file_size, srcnn_size;
> +
> +/// Check specified confguration file name and read network weights 
> from it
> +if (!srcnn_context->config_file_path){
> +av_log(context, AV_LOG_INFO, "configuration file for network was 
> not specified, using default weights for x2 upsampling\n");
> +
> +/// Create convolution kernels and copy default weights
> +srcnn_context->conv1.input_channels = 1;
> +srcnn_context->conv1.output_channels = 64;
> +srcnn_context->conv1.size = 9;
> +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv1, 
> conv1_kernel, conv1_biases), )
> +
> +srcnn_context->conv2.input_channels = 64;
> +srcnn_context->conv2.output_channels = 32;
> +srcnn_context->conv2.size = 1;
> +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv2, 
> conv2_kernel, conv2_biases), )
> +
> +srcnn_context->conv3.input_channels = 32;
> +srcnn_context->conv3.output_channels = 1;
> +srcnn_context->conv3.size = 5;
> +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv3, 
> conv3_kernel, conv3_biases), )
> +}
> +else if (access(srcnn_context->config_file_path, R_OK) != -1){

 This seems to be the function needed from unistd.h

 Looking at configure and libavformat/file.c it's apparently not
 available everywhere (we have a HAVE_ACCESS define, and also check for
 mode constants like R_OK).
>>>
>>> MSVC does not have access(), so i have temporarily made this filter
>>> depend on its presence until a proper fix is applied. That target has
>>> been broken for a few days now and that shouldn't be the case.
>>> It should be a matter or doing the same thing as in libavformat/file.c,
>>> i guess, or perhaps use _access() instead, which is available in MSVC.
>> Thanks.
>>
>>
>> @Sergey can you provide a definitive fix?
>>
>>> ___
>>> ffmpeg-devel mailing list
>>> ffmpeg-devel@ffmpeg.org
>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> ___
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> 

Re: [FFmpeg-devel] [GSOC] [PATCH] SRCNN filter

[Steven Liu]

Hi Sergey,

How should i test this filter?
I tested it some days ago, the picture get worse from 2nd frame.
input resolution 640x480 to 1280x720;

ffmpeg -i input -vf srcnn output

> On May 7, 2018, at 09:33, Pedro Arthur  wrote:
> 
> 2018-05-06 17:27 GMT-03:00 James Almer :
>> On 5/5/2018 5:38 PM, James Almer wrote:
>>> On 4/10/2018 2:16 PM, Sergey Lavrushkin wrote:
 diff --git a/libavfilter/vf_srcnn.c b/libavfilter/vf_srcnn.c
 new file mode 100644
 index 00..d9b4891f7f
 --- /dev/null
 +++ b/libavfilter/vf_srcnn.c
 @@ -0,0 +1,420 @@
 +/*
 + * Copyright (c) 2018 Sergey Lavrushkin
 + *
 + * This file is part of FFmpeg.
 + *
 + * FFmpeg is free software; you can redistribute it and/or
 + * modify it under the terms of the GNU Lesser General Public
 + * License as published by the Free Software Foundation; either
 + * version 2.1 of the License, or (at your option) any later version.
 + *
 + * FFmpeg is distributed in the hope that it will be useful,
 + * but WITHOUT ANY WARRANTY; without even the implied warranty of
 + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 + * Lesser General Public License for more details.
 + *
 + * You should have received a copy of the GNU Lesser General Public
 + * License along with FFmpeg; if not, write to the Free Software
 + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 
 02110-1301 USA
 + */
 +
 +/**
 + * @file
 + * Filter implementing image super-resolution using deep convolutional 
 networks.
 + * https://arxiv.org/abs/1501.00092
 + */
 +
 +#include "avfilter.h"
 +#include "formats.h"
 +#include "internal.h"
 +#include "libavutil/opt.h"
 +#include "unistd.h"
>>> 
>>> This broke msvc. It should be (if anything) , and wrapped
>>> inside a HAVE_UNISTD_H preprocessor check.
>> 
>> I (accidentally) applied this change as part of an unrelated commit, so
>> that's dealt with at least.
>> 
>>> 
 +static av_cold int init(AVFilterContext* context)
 +{
 +SRCNNContext *srcnn_context = context->priv;
 +AVIOContext* config_file_context;
 +int64_t file_size, srcnn_size;
 +
 +/// Check specified confguration file name and read network weights 
 from it
 +if (!srcnn_context->config_file_path){
 +av_log(context, AV_LOG_INFO, "configuration file for network was 
 not specified, using default weights for x2 upsampling\n");
 +
 +/// Create convolution kernels and copy default weights
 +srcnn_context->conv1.input_channels = 1;
 +srcnn_context->conv1.output_channels = 64;
 +srcnn_context->conv1.size = 9;
 +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv1, 
 conv1_kernel, conv1_biases), )
 +
 +srcnn_context->conv2.input_channels = 64;
 +srcnn_context->conv2.output_channels = 32;
 +srcnn_context->conv2.size = 1;
 +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv2, 
 conv2_kernel, conv2_biases), )
 +
 +srcnn_context->conv3.input_channels = 32;
 +srcnn_context->conv3.output_channels = 1;
 +srcnn_context->conv3.size = 5;
 +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv3, 
 conv3_kernel, conv3_biases), )
 +}
 +else if (access(srcnn_context->config_file_path, R_OK) != -1){
>>> 
>>> This seems to be the function needed from unistd.h
>>> 
>>> Looking at configure and libavformat/file.c it's apparently not
>>> available everywhere (we have a HAVE_ACCESS define, and also check for
>>> mode constants like R_OK).
>> 
>> MSVC does not have access(), so i have temporarily made this filter
>> depend on its presence until a proper fix is applied. That target has
>> been broken for a few days now and that shouldn't be the case.
>> It should be a matter or doing the same thing as in libavformat/file.c,
>> i guess, or perhaps use _access() instead, which is available in MSVC.
> Thanks.
> 
> 
> @Sergey can you provide a definitive fix?
> 
>> ___
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Thanks
Steven



___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [GSOC] [PATCH] SRCNN filter

[Pedro Arthur]

2018-05-06 17:27 GMT-03:00 James Almer :
> On 5/5/2018 5:38 PM, James Almer wrote:
>> On 4/10/2018 2:16 PM, Sergey Lavrushkin wrote:
>>> diff --git a/libavfilter/vf_srcnn.c b/libavfilter/vf_srcnn.c
>>> new file mode 100644
>>> index 00..d9b4891f7f
>>> --- /dev/null
>>> +++ b/libavfilter/vf_srcnn.c
>>> @@ -0,0 +1,420 @@
>>> +/*
>>> + * Copyright (c) 2018 Sergey Lavrushkin
>>> + *
>>> + * This file is part of FFmpeg.
>>> + *
>>> + * FFmpeg is free software; you can redistribute it and/or
>>> + * modify it under the terms of the GNU Lesser General Public
>>> + * License as published by the Free Software Foundation; either
>>> + * version 2.1 of the License, or (at your option) any later version.
>>> + *
>>> + * FFmpeg is distributed in the hope that it will be useful,
>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>>> + * Lesser General Public License for more details.
>>> + *
>>> + * You should have received a copy of the GNU Lesser General Public
>>> + * License along with FFmpeg; if not, write to the Free Software
>>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 
>>> 02110-1301 USA
>>> + */
>>> +
>>> +/**
>>> + * @file
>>> + * Filter implementing image super-resolution using deep convolutional 
>>> networks.
>>> + * https://arxiv.org/abs/1501.00092
>>> + */
>>> +
>>> +#include "avfilter.h"
>>> +#include "formats.h"
>>> +#include "internal.h"
>>> +#include "libavutil/opt.h"
>>> +#include "unistd.h"
>>
>> This broke msvc. It should be (if anything) , and wrapped
>> inside a HAVE_UNISTD_H preprocessor check.
>
> I (accidentally) applied this change as part of an unrelated commit, so
> that's dealt with at least.
>
>>
>>> +static av_cold int init(AVFilterContext* context)
>>> +{
>>> +SRCNNContext *srcnn_context = context->priv;
>>> +AVIOContext* config_file_context;
>>> +int64_t file_size, srcnn_size;
>>> +
>>> +/// Check specified confguration file name and read network weights 
>>> from it
>>> +if (!srcnn_context->config_file_path){
>>> +av_log(context, AV_LOG_INFO, "configuration file for network was 
>>> not specified, using default weights for x2 upsampling\n");
>>> +
>>> +/// Create convolution kernels and copy default weights
>>> +srcnn_context->conv1.input_channels = 1;
>>> +srcnn_context->conv1.output_channels = 64;
>>> +srcnn_context->conv1.size = 9;
>>> +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv1, 
>>> conv1_kernel, conv1_biases), )
>>> +
>>> +srcnn_context->conv2.input_channels = 64;
>>> +srcnn_context->conv2.output_channels = 32;
>>> +srcnn_context->conv2.size = 1;
>>> +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv2, 
>>> conv2_kernel, conv2_biases), )
>>> +
>>> +srcnn_context->conv3.input_channels = 32;
>>> +srcnn_context->conv3.output_channels = 1;
>>> +srcnn_context->conv3.size = 5;
>>> +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv3, 
>>> conv3_kernel, conv3_biases), )
>>> +}
>>> +else if (access(srcnn_context->config_file_path, R_OK) != -1){
>>
>> This seems to be the function needed from unistd.h
>>
>> Looking at configure and libavformat/file.c it's apparently not
>> available everywhere (we have a HAVE_ACCESS define, and also check for
>> mode constants like R_OK).
>
> MSVC does not have access(), so i have temporarily made this filter
> depend on its presence until a proper fix is applied. That target has
> been broken for a few days now and that shouldn't be the case.
> It should be a matter or doing the same thing as in libavformat/file.c,
> i guess, or perhaps use _access() instead, which is available in MSVC.
Thanks.


@Sergey can you provide a definitive fix?

> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [GSOC] [PATCH] SRCNN filter

[James Almer]

On 5/5/2018 5:38 PM, James Almer wrote:
> On 4/10/2018 2:16 PM, Sergey Lavrushkin wrote:
>> diff --git a/libavfilter/vf_srcnn.c b/libavfilter/vf_srcnn.c
>> new file mode 100644
>> index 00..d9b4891f7f
>> --- /dev/null
>> +++ b/libavfilter/vf_srcnn.c
>> @@ -0,0 +1,420 @@
>> +/*
>> + * Copyright (c) 2018 Sergey Lavrushkin
>> + *
>> + * This file is part of FFmpeg.
>> + *
>> + * FFmpeg is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2.1 of the License, or (at your option) any later version.
>> + *
>> + * FFmpeg is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> + * Lesser General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public
>> + * License along with FFmpeg; if not, write to the Free Software
>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 
>> USA
>> + */
>> +
>> +/**
>> + * @file
>> + * Filter implementing image super-resolution using deep convolutional 
>> networks.
>> + * https://arxiv.org/abs/1501.00092
>> + */
>> +
>> +#include "avfilter.h"
>> +#include "formats.h"
>> +#include "internal.h"
>> +#include "libavutil/opt.h"
>> +#include "unistd.h"
> 
> This broke msvc. It should be (if anything) , and wrapped
> inside a HAVE_UNISTD_H preprocessor check.

I (accidentally) applied this change as part of an unrelated commit, so
that's dealt with at least.

> 
>> +static av_cold int init(AVFilterContext* context)
>> +{
>> +SRCNNContext *srcnn_context = context->priv;
>> +AVIOContext* config_file_context;
>> +int64_t file_size, srcnn_size;
>> +
>> +/// Check specified confguration file name and read network weights 
>> from it
>> +if (!srcnn_context->config_file_path){
>> +av_log(context, AV_LOG_INFO, "configuration file for network was 
>> not specified, using default weights for x2 upsampling\n");
>> +
>> +/// Create convolution kernels and copy default weights
>> +srcnn_context->conv1.input_channels = 1;
>> +srcnn_context->conv1.output_channels = 64;
>> +srcnn_context->conv1.size = 9;
>> +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv1, 
>> conv1_kernel, conv1_biases), )
>> +
>> +srcnn_context->conv2.input_channels = 64;
>> +srcnn_context->conv2.output_channels = 32;
>> +srcnn_context->conv2.size = 1;
>> +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv2, 
>> conv2_kernel, conv2_biases), )
>> +
>> +srcnn_context->conv3.input_channels = 32;
>> +srcnn_context->conv3.output_channels = 1;
>> +srcnn_context->conv3.size = 5;
>> +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv3, 
>> conv3_kernel, conv3_biases), )
>> +}
>> +else if (access(srcnn_context->config_file_path, R_OK) != -1){
> 
> This seems to be the function needed from unistd.h
> 
> Looking at configure and libavformat/file.c it's apparently not
> available everywhere (we have a HAVE_ACCESS define, and also check for
> mode constants like R_OK).

MSVC does not have access(), so i have temporarily made this filter
depend on its presence until a proper fix is applied. That target has
been broken for a few days now and that shouldn't be the case.
It should be a matter or doing the same thing as in libavformat/file.c,
i guess, or perhaps use _access() instead, which is available in MSVC.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [GSOC] [PATCH] SRCNN filter

[James Almer]

On 4/10/2018 2:16 PM, Sergey Lavrushkin wrote:
> diff --git a/libavfilter/vf_srcnn.c b/libavfilter/vf_srcnn.c
> new file mode 100644
> index 00..d9b4891f7f
> --- /dev/null
> +++ b/libavfilter/vf_srcnn.c
> @@ -0,0 +1,420 @@
> +/*
> + * Copyright (c) 2018 Sergey Lavrushkin
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 
> USA
> + */
> +
> +/**
> + * @file
> + * Filter implementing image super-resolution using deep convolutional 
> networks.
> + * https://arxiv.org/abs/1501.00092
> + */
> +
> +#include "avfilter.h"
> +#include "formats.h"
> +#include "internal.h"
> +#include "libavutil/opt.h"
> +#include "unistd.h"

This broke msvc. It should be (if anything) , and wrapped
inside a HAVE_UNISTD_H preprocessor check.

> +static av_cold int init(AVFilterContext* context)
> +{
> +SRCNNContext *srcnn_context = context->priv;
> +AVIOContext* config_file_context;
> +int64_t file_size, srcnn_size;
> +
> +/// Check specified confguration file name and read network weights from 
> it
> +if (!srcnn_context->config_file_path){
> +av_log(context, AV_LOG_INFO, "configuration file for network was not 
> specified, using default weights for x2 upsampling\n");
> +
> +/// Create convolution kernels and copy default weights
> +srcnn_context->conv1.input_channels = 1;
> +srcnn_context->conv1.output_channels = 64;
> +srcnn_context->conv1.size = 9;
> +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv1, 
> conv1_kernel, conv1_biases), )
> +
> +srcnn_context->conv2.input_channels = 64;
> +srcnn_context->conv2.output_channels = 32;
> +srcnn_context->conv2.size = 1;
> +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv2, 
> conv2_kernel, conv2_biases), )
> +
> +srcnn_context->conv3.input_channels = 32;
> +srcnn_context->conv3.output_channels = 1;
> +srcnn_context->conv3.size = 5;
> +CHECK_ALLOCATION(allocate_copy_conv_data(_context->conv3, 
> conv3_kernel, conv3_biases), )
> +}
> +else if (access(srcnn_context->config_file_path, R_OK) != -1){

This seems to be the function needed from unistd.h

Looking at configure and libavformat/file.c it's apparently not
available everywhere (we have a HAVE_ACCESS define, and also check for
mode constants like R_OK).
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [GSOC] [PATCH] SRCNN filter

[Pedro Arthur]

2018-05-03 17:58 GMT-03:00 Michael Niedermayer :
> On Thu, May 03, 2018 at 03:17:11PM -0300, Pedro Arthur wrote:
>> 2018-04-10 14:16 GMT-03:00 Sergey Lavrushkin :
>> > 2018-03-29 3:55 GMT+03:00 Michael Niedermayer :
>> >
>> >> On Wed, Mar 28, 2018 at 11:17:40AM +0300, Sergey Lavrushkin wrote:
>> >> > > [...]
>> >> > > > +#define OFFSET(x) offsetof(SRCNNContext, x)
>> >> > > > +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM | AV_OPT_FLAG_VIDEO_PARAM
>> >> > > > +static const AVOption srcnn_options[] = {
>> >> > > > +{ "config_file", "path to configuration file with network
>> >> > > parameters", OFFSET(config_file_path), AV_OPT_TYPE_STRING,
>> >> {.str=NULL}, 0,
>> >> > > 0, FLAGS },
>> >> > > > +{ NULL }
>> >> > > > +};
>> >> > > > +
>> >> > > > +AVFILTER_DEFINE_CLASS(srcnn);
>> >> > > > +
>> >> > > > +#define CHECK_FILE(file)if (ferror(file) || feof(file)){ \
>> >> > > > +av_log(context, AV_LOG_ERROR, 
>> >> > > > "error
>> >> > > reading configuration file\n");\
>> >> > > > +fclose(file); \
>> >> > > > +return AVERROR(EIO); \
>> >> > > > +}
>> >> > > > +
>> >> > > > +#define CHECK_ALLOCATION(conv, file)if
>> >> > > (allocate_and_read_convolution_data(, file)){ \
>> >> > > > +av_log(context,
>> >> > > AV_LOG_ERROR, "could not allocate memory for convolutions\n"); \
>> >> > > > +fclose(file); \
>> >> > > > +return AVERROR(ENOMEM);
>> >> \
>> >> > > > +}
>> >> > > > +
>> >> > >
>> >> > > > +static int allocate_and_read_convolution_data(Convolution* conv,
>> >> FILE*
>> >> > > config_file)
>> >> > > > +{
>> >> > > > +int32_t kernel_size = conv->output_channels * conv->size *
>> >> > > conv->size * conv->input_channels;
>> >> > > > +conv->kernel = av_malloc(kernel_size * sizeof(double));
>> >> > > > +if (!conv->kernel){
>> >> > > > +return 1;
>> >> > >
>> >> > > this should return an AVERROR code for consistency with the rest of
>> >> > > the codebase
>> >> > >
>> >> >
>> >> > Ok.
>> >> >
>> >> >
>> >> > > > +}
>> >> > >
>> >> > > > +fread(conv->kernel, sizeof(double), kernel_size, config_file);
>> >> > >
>> >> > > directly reading data types is not portable, it would for example be
>> >> > > endian specific
>> >> > > and using avio for reading may be better, though fread is as far as 
>> >> > > iam
>> >> > > concerned also ok
>> >> > >
>> >> >
>> >> > Ok, I understand the problem, but I have not really worked with it
>> >> before,
>> >> > so I need an advice of how to properly fix it. If I understand 
>> >> > correctly,
>> >> > for
>> >>
>> >> > int32_t I need to check endianness and reverse bytes if necessary. But
>> >> with
>> >>
>> >> see avio_rb32()
>> >> might not be as fast as reading a whole array and byteswaping though.
>> >> Not sure it matters
>> >>
>> >>
>> >> > doubles it is more complicated. Should I write a IEEE 754 converter from
>> >> > binary
>> >> > to double or maybe I can somehow check IEEE 754 doubles support and
>> >> > depending
>> >> > on it either stick to the default network weights, or just read bytes 
>> >> > and
>> >> > check
>> >> > endianness, if IEEE 754 doubles are supported? Or maybe avio provide 
>> >> > some
>> >> > utility to deal with this problem?
>> >>
>> >> something like this should work:
>> >> av_int2float(avio_rb32(pb));
>> >> av_int2double(avio_rb64(pb));
>> >>
>> >> >
>> >> >
>> >> > > [...]
>> >> > > > +/**
>> >> > > > + * @file
>> >> > > > + * Default cnn weights for x2 upsampling with srcnn filter.
>> >> > > > + */
>> >> > > > +
>> >> > > > +/// First convolution kernel
>> >> > >
>> >> > > > +static double conv1_kernel[] = {
>> >> > >
>> >> > > static data should be also const, otherwise it may be changed and 
>> >> > > could
>> >> > > cause
>> >> > > thread saftey issues
>> >> > >
>> >> >
>> >> > Ok, I just wanted to not allocate additional memory in case of using
>> >> > default weights.
>> >>
>> >> well, there can be more than one instance of a filter running at the same
>> >> time
>> >> so static variables that are actually changing will affect the other
>> >> filter instance
>> >>
>> >
>> > Hi,
>> >
>> > Here is the updated patch with fixed issues.
>> > Sorry for the late reply, I was busy doing things related to my university
>> > activities.
>> >
>> > ___
>> > ffmpeg-devel mailing list
>> > ffmpeg-devel@ffmpeg.org
>> > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> >
>>
>> I think all concerns were addressed, can the patch be pushed?
>
> sure you can push it if it looks ok to you.
> please make sure fate-source passes (i think it fails with the patch)
Fixed and pushed.

>
> thanks
>
> [...]
> --
> Michael 

Re: [FFmpeg-devel] [GSOC] [PATCH] SRCNN filter

[Michael Niedermayer]

On Thu, May 03, 2018 at 03:17:11PM -0300, Pedro Arthur wrote:
> 2018-04-10 14:16 GMT-03:00 Sergey Lavrushkin :
> > 2018-03-29 3:55 GMT+03:00 Michael Niedermayer :
> >
> >> On Wed, Mar 28, 2018 at 11:17:40AM +0300, Sergey Lavrushkin wrote:
> >> > > [...]
> >> > > > +#define OFFSET(x) offsetof(SRCNNContext, x)
> >> > > > +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM | AV_OPT_FLAG_VIDEO_PARAM
> >> > > > +static const AVOption srcnn_options[] = {
> >> > > > +{ "config_file", "path to configuration file with network
> >> > > parameters", OFFSET(config_file_path), AV_OPT_TYPE_STRING,
> >> {.str=NULL}, 0,
> >> > > 0, FLAGS },
> >> > > > +{ NULL }
> >> > > > +};
> >> > > > +
> >> > > > +AVFILTER_DEFINE_CLASS(srcnn);
> >> > > > +
> >> > > > +#define CHECK_FILE(file)if (ferror(file) || feof(file)){ \
> >> > > > +av_log(context, AV_LOG_ERROR, "error
> >> > > reading configuration file\n");\
> >> > > > +fclose(file); \
> >> > > > +return AVERROR(EIO); \
> >> > > > +}
> >> > > > +
> >> > > > +#define CHECK_ALLOCATION(conv, file)if
> >> > > (allocate_and_read_convolution_data(, file)){ \
> >> > > > +av_log(context,
> >> > > AV_LOG_ERROR, "could not allocate memory for convolutions\n"); \
> >> > > > +fclose(file); \
> >> > > > +return AVERROR(ENOMEM);
> >> \
> >> > > > +}
> >> > > > +
> >> > >
> >> > > > +static int allocate_and_read_convolution_data(Convolution* conv,
> >> FILE*
> >> > > config_file)
> >> > > > +{
> >> > > > +int32_t kernel_size = conv->output_channels * conv->size *
> >> > > conv->size * conv->input_channels;
> >> > > > +conv->kernel = av_malloc(kernel_size * sizeof(double));
> >> > > > +if (!conv->kernel){
> >> > > > +return 1;
> >> > >
> >> > > this should return an AVERROR code for consistency with the rest of
> >> > > the codebase
> >> > >
> >> >
> >> > Ok.
> >> >
> >> >
> >> > > > +}
> >> > >
> >> > > > +fread(conv->kernel, sizeof(double), kernel_size, config_file);
> >> > >
> >> > > directly reading data types is not portable, it would for example be
> >> > > endian specific
> >> > > and using avio for reading may be better, though fread is as far as iam
> >> > > concerned also ok
> >> > >
> >> >
> >> > Ok, I understand the problem, but I have not really worked with it
> >> before,
> >> > so I need an advice of how to properly fix it. If I understand correctly,
> >> > for
> >>
> >> > int32_t I need to check endianness and reverse bytes if necessary. But
> >> with
> >>
> >> see avio_rb32()
> >> might not be as fast as reading a whole array and byteswaping though.
> >> Not sure it matters
> >>
> >>
> >> > doubles it is more complicated. Should I write a IEEE 754 converter from
> >> > binary
> >> > to double or maybe I can somehow check IEEE 754 doubles support and
> >> > depending
> >> > on it either stick to the default network weights, or just read bytes and
> >> > check
> >> > endianness, if IEEE 754 doubles are supported? Or maybe avio provide some
> >> > utility to deal with this problem?
> >>
> >> something like this should work:
> >> av_int2float(avio_rb32(pb));
> >> av_int2double(avio_rb64(pb));
> >>
> >> >
> >> >
> >> > > [...]
> >> > > > +/**
> >> > > > + * @file
> >> > > > + * Default cnn weights for x2 upsampling with srcnn filter.
> >> > > > + */
> >> > > > +
> >> > > > +/// First convolution kernel
> >> > >
> >> > > > +static double conv1_kernel[] = {
> >> > >
> >> > > static data should be also const, otherwise it may be changed and could
> >> > > cause
> >> > > thread saftey issues
> >> > >
> >> >
> >> > Ok, I just wanted to not allocate additional memory in case of using
> >> > default weights.
> >>
> >> well, there can be more than one instance of a filter running at the same
> >> time
> >> so static variables that are actually changing will affect the other
> >> filter instance
> >>
> >
> > Hi,
> >
> > Here is the updated patch with fixed issues.
> > Sorry for the late reply, I was busy doing things related to my university
> > activities.
> >
> > ___
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> 
> I think all concerns were addressed, can the patch be pushed?

sure you can push it if it looks ok to you.
please make sure fate-source passes (i think it fails with the patch)

thanks

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Into a blind darkness they enter who follow after the Ignorance,
they as if into a greater darkness enter who devote themselves
to the Knowledge alone. -- Isha Upanishad


signature.asc
Description: PGP signature

Re: [FFmpeg-devel] [GSOC] [PATCH] SRCNN filter

[Pedro Arthur]

2018-04-10 14:16 GMT-03:00 Sergey Lavrushkin :
> 2018-03-29 3:55 GMT+03:00 Michael Niedermayer :
>
>> On Wed, Mar 28, 2018 at 11:17:40AM +0300, Sergey Lavrushkin wrote:
>> > > [...]
>> > > > +#define OFFSET(x) offsetof(SRCNNContext, x)
>> > > > +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM | AV_OPT_FLAG_VIDEO_PARAM
>> > > > +static const AVOption srcnn_options[] = {
>> > > > +{ "config_file", "path to configuration file with network
>> > > parameters", OFFSET(config_file_path), AV_OPT_TYPE_STRING,
>> {.str=NULL}, 0,
>> > > 0, FLAGS },
>> > > > +{ NULL }
>> > > > +};
>> > > > +
>> > > > +AVFILTER_DEFINE_CLASS(srcnn);
>> > > > +
>> > > > +#define CHECK_FILE(file)if (ferror(file) || feof(file)){ \
>> > > > +av_log(context, AV_LOG_ERROR, "error
>> > > reading configuration file\n");\
>> > > > +fclose(file); \
>> > > > +return AVERROR(EIO); \
>> > > > +}
>> > > > +
>> > > > +#define CHECK_ALLOCATION(conv, file)if
>> > > (allocate_and_read_convolution_data(, file)){ \
>> > > > +av_log(context,
>> > > AV_LOG_ERROR, "could not allocate memory for convolutions\n"); \
>> > > > +fclose(file); \
>> > > > +return AVERROR(ENOMEM);
>> \
>> > > > +}
>> > > > +
>> > >
>> > > > +static int allocate_and_read_convolution_data(Convolution* conv,
>> FILE*
>> > > config_file)
>> > > > +{
>> > > > +int32_t kernel_size = conv->output_channels * conv->size *
>> > > conv->size * conv->input_channels;
>> > > > +conv->kernel = av_malloc(kernel_size * sizeof(double));
>> > > > +if (!conv->kernel){
>> > > > +return 1;
>> > >
>> > > this should return an AVERROR code for consistency with the rest of
>> > > the codebase
>> > >
>> >
>> > Ok.
>> >
>> >
>> > > > +}
>> > >
>> > > > +fread(conv->kernel, sizeof(double), kernel_size, config_file);
>> > >
>> > > directly reading data types is not portable, it would for example be
>> > > endian specific
>> > > and using avio for reading may be better, though fread is as far as iam
>> > > concerned also ok
>> > >
>> >
>> > Ok, I understand the problem, but I have not really worked with it
>> before,
>> > so I need an advice of how to properly fix it. If I understand correctly,
>> > for
>>
>> > int32_t I need to check endianness and reverse bytes if necessary. But
>> with
>>
>> see avio_rb32()
>> might not be as fast as reading a whole array and byteswaping though.
>> Not sure it matters
>>
>>
>> > doubles it is more complicated. Should I write a IEEE 754 converter from
>> > binary
>> > to double or maybe I can somehow check IEEE 754 doubles support and
>> > depending
>> > on it either stick to the default network weights, or just read bytes and
>> > check
>> > endianness, if IEEE 754 doubles are supported? Or maybe avio provide some
>> > utility to deal with this problem?
>>
>> something like this should work:
>> av_int2float(avio_rb32(pb));
>> av_int2double(avio_rb64(pb));
>>
>> >
>> >
>> > > [...]
>> > > > +/**
>> > > > + * @file
>> > > > + * Default cnn weights for x2 upsampling with srcnn filter.
>> > > > + */
>> > > > +
>> > > > +/// First convolution kernel
>> > >
>> > > > +static double conv1_kernel[] = {
>> > >
>> > > static data should be also const, otherwise it may be changed and could
>> > > cause
>> > > thread saftey issues
>> > >
>> >
>> > Ok, I just wanted to not allocate additional memory in case of using
>> > default weights.
>>
>> well, there can be more than one instance of a filter running at the same
>> time
>> so static variables that are actually changing will affect the other
>> filter instance
>>
>
> Hi,
>
> Here is the updated patch with fixed issues.
> Sorry for the late reply, I was busy doing things related to my university
> activities.
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>

I think all concerns were addressed, can the patch be pushed?
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [GSOC] [PATCH] SRCNN filter

[Pedro Arthur]

Thanks!
Please send it to the same mailing list thread where it was being reviewd.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [GSOC] [PATCH] SRCNN filter

[Michael Niedermayer]

On Wed, Mar 28, 2018 at 11:17:40AM +0300, Sergey Lavrushkin wrote:
> > [...]
> > > +#define OFFSET(x) offsetof(SRCNNContext, x)
> > > +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM | AV_OPT_FLAG_VIDEO_PARAM
> > > +static const AVOption srcnn_options[] = {
> > > +{ "config_file", "path to configuration file with network
> > parameters", OFFSET(config_file_path), AV_OPT_TYPE_STRING, {.str=NULL}, 0,
> > 0, FLAGS },
> > > +{ NULL }
> > > +};
> > > +
> > > +AVFILTER_DEFINE_CLASS(srcnn);
> > > +
> > > +#define CHECK_FILE(file)if (ferror(file) || feof(file)){ \
> > > +av_log(context, AV_LOG_ERROR, "error
> > reading configuration file\n");\
> > > +fclose(file); \
> > > +return AVERROR(EIO); \
> > > +}
> > > +
> > > +#define CHECK_ALLOCATION(conv, file)if
> > (allocate_and_read_convolution_data(, file)){ \
> > > +av_log(context,
> > AV_LOG_ERROR, "could not allocate memory for convolutions\n"); \
> > > +fclose(file); \
> > > +return AVERROR(ENOMEM); \
> > > +}
> > > +
> >
> > > +static int allocate_and_read_convolution_data(Convolution* conv, FILE*
> > config_file)
> > > +{
> > > +int32_t kernel_size = conv->output_channels * conv->size *
> > conv->size * conv->input_channels;
> > > +conv->kernel = av_malloc(kernel_size * sizeof(double));
> > > +if (!conv->kernel){
> > > +return 1;
> >
> > this should return an AVERROR code for consistency with the rest of
> > the codebase
> >
> 
> Ok.
> 
> 
> > > +}
> >
> > > +fread(conv->kernel, sizeof(double), kernel_size, config_file);
> >
> > directly reading data types is not portable, it would for example be
> > endian specific
> > and using avio for reading may be better, though fread is as far as iam
> > concerned also ok
> >
> 
> Ok, I understand the problem, but I have not really worked with it before,
> so I need an advice of how to properly fix it. If I understand correctly,
> for

> int32_t I need to check endianness and reverse bytes if necessary. But with

see avio_rb32()
might not be as fast as reading a whole array and byteswaping though.
Not sure it matters


> doubles it is more complicated. Should I write a IEEE 754 converter from
> binary
> to double or maybe I can somehow check IEEE 754 doubles support and
> depending
> on it either stick to the default network weights, or just read bytes and
> check
> endianness, if IEEE 754 doubles are supported? Or maybe avio provide some
> utility to deal with this problem?

something like this should work:
av_int2float(avio_rb32(pb));
av_int2double(avio_rb64(pb));

> 
> 
> > [...]
> > > +/**
> > > + * @file
> > > + * Default cnn weights for x2 upsampling with srcnn filter.
> > > + */
> > > +
> > > +/// First convolution kernel
> >
> > > +static double conv1_kernel[] = {
> >
> > static data should be also const, otherwise it may be changed and could
> > cause
> > thread saftey issues
> >
> 
> Ok, I just wanted to not allocate additional memory in case of using
> default weights.

well, there can be more than one instance of a filter running at the same time
so static variables that are actually changing will affect the other
filter instance


[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

When the tyrant has disposed of foreign enemies by conquest or treaty, and
there is nothing more to fear from them, then he is always stirring up
some war or other, in order that the people may require a leader. -- Plato


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [GSOC] [PATCH] SRCNN filter

[Sergey Lavrushkin]

> [...]
> > +#define OFFSET(x) offsetof(SRCNNContext, x)
> > +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM | AV_OPT_FLAG_VIDEO_PARAM
> > +static const AVOption srcnn_options[] = {
> > +{ "config_file", "path to configuration file with network
> parameters", OFFSET(config_file_path), AV_OPT_TYPE_STRING, {.str=NULL}, 0,
> 0, FLAGS },
> > +{ NULL }
> > +};
> > +
> > +AVFILTER_DEFINE_CLASS(srcnn);
> > +
> > +#define CHECK_FILE(file)if (ferror(file) || feof(file)){ \
> > +av_log(context, AV_LOG_ERROR, "error
> reading configuration file\n");\
> > +fclose(file); \
> > +return AVERROR(EIO); \
> > +}
> > +
> > +#define CHECK_ALLOCATION(conv, file)if
> (allocate_and_read_convolution_data(, file)){ \
> > +av_log(context,
> AV_LOG_ERROR, "could not allocate memory for convolutions\n"); \
> > +fclose(file); \
> > +return AVERROR(ENOMEM); \
> > +}
> > +
>
> > +static int allocate_and_read_convolution_data(Convolution* conv, FILE*
> config_file)
> > +{
> > +int32_t kernel_size = conv->output_channels * conv->size *
> conv->size * conv->input_channels;
> > +conv->kernel = av_malloc(kernel_size * sizeof(double));
> > +if (!conv->kernel){
> > +return 1;
>
> this should return an AVERROR code for consistency with the rest of
> the codebase
>

Ok.


> > +}
>
> > +fread(conv->kernel, sizeof(double), kernel_size, config_file);
>
> directly reading data types is not portable, it would for example be
> endian specific
> and using avio for reading may be better, though fread is as far as iam
> concerned also ok
>

Ok, I understand the problem, but I have not really worked with it before,
so I need an advice of how to properly fix it. If I understand correctly,
for
int32_t I need to check endianness and reverse bytes if necessary. But with
doubles it is more complicated. Should I write a IEEE 754 converter from
binary
to double or maybe I can somehow check IEEE 754 doubles support and
depending
on it either stick to the default network weights, or just read bytes and
check
endianness, if IEEE 754 doubles are supported? Or maybe avio provide some
utility to deal with this problem?


> [...]
> > +/**
> > + * @file
> > + * Default cnn weights for x2 upsampling with srcnn filter.
> > + */
> > +
> > +/// First convolution kernel
>
> > +static double conv1_kernel[] = {
>
> static data should be also const, otherwise it may be changed and could
> cause
> thread saftey issues
>

Ok, I just wanted to not allocate additional memory in case of using
default weights.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [GSOC] [PATCH] SRCNN filter

[Michael Niedermayer]

On Sun, Mar 25, 2018 at 08:04:17PM +0300, Sergey Lavrushkin wrote:
> Hello,
> 
> This is my implementation of qualification task for GSOC Super Resolution
> Filter project.
> For default x2 upsampling with provided srcnn filter following command can
> be used:
> ffmpeg -i input -vf scale=iw*2:-1,srcnn output
> Also other models for x2, x3, x4 upsampling can be specified using
> following command:
> ffmpeg -i input -vf scale=iw*factor:-1,srcnn=path_to_model output
> Configuration files with other models can be found here:
> https://drive.google.com/drive/folders/1-M9azWTtZ4egf8ndRU7Y
> _tiGP6QtN-Fp?usp=sharing
> And here are some examples of its work:
> https://drive.google.com/drive/folders/1jMVMKnre2KG0avq2zNw6
> CMqUKdPWnlQo?usp=sharing

[...]
> +#define OFFSET(x) offsetof(SRCNNContext, x)
> +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM | AV_OPT_FLAG_VIDEO_PARAM
> +static const AVOption srcnn_options[] = {
> +{ "config_file", "path to configuration file with network parameters", 
> OFFSET(config_file_path), AV_OPT_TYPE_STRING, {.str=NULL}, 0, 0, FLAGS },
> +{ NULL }
> +};
> +
> +AVFILTER_DEFINE_CLASS(srcnn);
> +
> +#define CHECK_FILE(file)if (ferror(file) || feof(file)){ \
> +av_log(context, AV_LOG_ERROR, "error reading 
> configuration file\n");\
> +fclose(file); \
> +return AVERROR(EIO); \
> +}
> +
> +#define CHECK_ALLOCATION(conv, file)if 
> (allocate_and_read_convolution_data(, file)){ \
> +av_log(context, AV_LOG_ERROR, 
> "could not allocate memory for convolutions\n"); \
> +fclose(file); \
> +return AVERROR(ENOMEM); \
> +}
> +

> +static int allocate_and_read_convolution_data(Convolution* conv, FILE* 
> config_file)
> +{
> +int32_t kernel_size = conv->output_channels * conv->size * conv->size * 
> conv->input_channels;
> +conv->kernel = av_malloc(kernel_size * sizeof(double));
> +if (!conv->kernel){
> +return 1;

this should return an AVERROR code for consistency with the rest of
the codebase


> +}

> +fread(conv->kernel, sizeof(double), kernel_size, config_file);

directly reading data types is not portable, it would for example be
endian specific
and using avio for reading may be better, though fread is as far as iam
concerned also ok

[...]
> +/**
> + * @file
> + * Default cnn weights for x2 upsampling with srcnn filter.
> + */
> +
> +/// First convolution kernel

> +static double conv1_kernel[] = {

static data should be also const, otherwise it may be changed and could cause
thread saftey issues

[...]

-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

You can kill me, but you cannot change the truth.


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel