Re: [FFmpeg-devel] [DISCUSSION] Motion Estimation API/Library

2017-08-04 Thread Alex Giladi 
I like the idea -- it will be then possible to link to hardware
acceleration.

I don't think R-D optimization (including taking VLC / CABAC into account)
belongs to the library -- this, as well as figuring out partitions, should
be up to the actual encoder.

On Thu, Aug 3, 2017 at 4:09 PM, Ronald S. Bultje  wrote:

> Hi,
>
> On Thu, Aug 3, 2017 at 5:32 PM, Davinder Singh 
> wrote:
>
> > On Tue, Aug 1, 2017 at 7:40 AM Davinder Singh 
> wrote:
> >
> > > [...]
> > >
> >
> > Keeping where the code lives, aside,
> >
> > main thing is API, so we need to talk about it.
>
>
> You're assuming that we know in advance what the API will have to do. I
> don't think that's the case.
>
> For example, what block sizes does it operate on? me_cc doesn't work for
> 64x64 blocks in HEVC, VP9 (and AV1?).
>
> Or what if we don't use VLC coding, but some arithmetic variation? Like all
> H.264, VP8-9, HEVC (and AV1?). Where perhaps the arithmetic coding can
> depend not just on intra/inter, but also on transform size, chroma/luma,
> etc.
>
> What if qscale is not uniform and allows per-coefficient matrix variations?
> How many sub-types of such variations will exist?
>
> My point is: designing it in advance for everything like this is insane.
> Just copy what is there, and prepare for the API to change _all the time_
> while we're molding it to fit in other problem shapes.
>
> Ronald
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [DISCUSSION] Motion Estimation API/Library

2017-08-03 Thread Ronald S. Bultje 
Hi,

On Thu, Aug 3, 2017 at 5:32 PM, Davinder Singh  wrote:

> On Tue, Aug 1, 2017 at 7:40 AM Davinder Singh  wrote:
>
> > [...]
> >
>
> Keeping where the code lives, aside,
>
> main thing is API, so we need to talk about it.


You're assuming that we know in advance what the API will have to do. I
don't think that's the case.

For example, what block sizes does it operate on? me_cc doesn't work for
64x64 blocks in HEVC, VP9 (and AV1?).

Or what if we don't use VLC coding, but some arithmetic variation? Like all
H.264, VP8-9, HEVC (and AV1?). Where perhaps the arithmetic coding can
depend not just on intra/inter, but also on transform size, chroma/luma,
etc.

What if qscale is not uniform and allows per-coefficient matrix variations?
How many sub-types of such variations will exist?

My point is: designing it in advance for everything like this is insane.
Just copy what is there, and prepare for the API to change _all the time_
while we're molding it to fit in other problem shapes.

Ronald
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [DISCUSSION] Motion Estimation API/Library

2017-08-03 Thread Davinder Singh 
On Tue, Aug 1, 2017 at 7:40 AM Davinder Singh  wrote:

> [...]
>

Keeping where the code lives, aside,

main thing is API, so we need to talk about it.
-- 
Davinder Singh
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [DISCUSSION] Motion Estimation API/Library

2017-08-01 Thread Ivan Kalvachev 
On 8/1/17, Clément Bœsch  wrote:
> On Tue, Aug 01, 2017 at 09:18:33AM +, Davinder Singh wrote:
> [...]
>> > In particular, the main policy of FFmpeg is to not depend on external
>> > libraries for core features. Therefore, if your project is a separate
>> >
>>
>> Just to be clear, it won't be "external" library like OpenCV...
>>
>>
>> > library, it will definitely not be used by FFmpeg codecs like you wish
>> > in your first paragraph.
>> >
>> > If you want a fighting chance of a project that is not stillborn, I
>> > think you need to make it part of FFmpeg, and make sure important
>> >
>>
>> .. it will be part of FFmpeg like libavfilter, just a new module -
>> libmotion.
>>
>
> yeah, but not a good idea. Just make it an optional component of
> libavutil.

Why not keep the code in libavcodec?

I think that (I)DCT was also given standard API and it is still there.
Also some me_cmp_fn use (I)DCT.

Best Regards
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [DISCUSSION] Motion Estimation API/Library

2017-08-01 Thread Rostislav Pehlivanov 
On 1 August 2017 at 14:45, Nicolas George  wrote:

> Le quartidi 14 thermidor, an CCXXV, Rostislav Pehlivanov a écrit :
> > I think I'd rather have it as a separate module. The part of the code
> could
> > grow to be quite big and the API quite complex depending on the needs of
> > encoders and filters.
>
> At this point in time, there are five C files and a couple of asm files
> per arch, I think it is premature to speculate about what it can become
> later. Plus, I do not really see the problem in having many files; a
> good naming discipline like for filters is all we need.
>
> On the other hand, right now, yet another library has many drawbacks:
> more complex and slower build, a name collision with a (rather obscure)
> Android library, more spaghetti dependencies for the users.
>
> I am with Clément on this issue, although I do not care about which
> component it gets in.
>
> Regards,
>
> --
>   Nicolas George
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>

Fair enough, though the API should be avpriv for now.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [DISCUSSION] Motion Estimation API/Library

2017-08-01 Thread Nicolas George 
Le quartidi 14 thermidor, an CCXXV, Rostislav Pehlivanov a écrit :
> I think I'd rather have it as a separate module. The part of the code could
> grow to be quite big and the API quite complex depending on the needs of
> encoders and filters.

At this point in time, there are five C files and a couple of asm files
per arch, I think it is premature to speculate about what it can become
later. Plus, I do not really see the problem in having many files; a
good naming discipline like for filters is all we need.

On the other hand, right now, yet another library has many drawbacks:
more complex and slower build, a name collision with a (rather obscure)
Android library, more spaghetti dependencies for the users.

I am with Clément on this issue, although I do not care about which
component it gets in.

Regards,

-- 
  Nicolas George


signature.asc
Description: Digital signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [DISCUSSION] Motion Estimation API/Library

2017-08-01 Thread Rostislav Pehlivanov 
On 1 August 2017 at 12:30, Clément Bœsch  wrote:

> On Tue, Aug 01, 2017 at 09:18:33AM +, Davinder Singh wrote:
> [...]
> > > In particular, the main policy of FFmpeg is to not depend on external
> > > libraries for core features. Therefore, if your project is a separate
> > >
> >
> > Just to be clear, it won't be "external" library like OpenCV...
> >
> >
> > > library, it will definitely not be used by FFmpeg codecs like you wish
> > > in your first paragraph.
> > >
> > > If you want a fighting chance of a project that is not stillborn, I
> > > think you need to make it part of FFmpeg, and make sure important
> > >
> >
> > .. it will be part of FFmpeg like libavfilter, just a new module -
> > libmotion.
> >
>
> yeah, but not a good idea. Just make it an optional component of
> libavutil.
>
> --
> Clément B.
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>
I think I'd rather have it as a separate module. The part of the code could
grow to be quite big and the API quite complex depending on the needs of
encoders and filters.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [DISCUSSION] Motion Estimation API/Library

2017-08-01 Thread Clément Bœsch 
On Tue, Aug 01, 2017 at 09:18:33AM +, Davinder Singh wrote:
[...]
> > In particular, the main policy of FFmpeg is to not depend on external
> > libraries for core features. Therefore, if your project is a separate
> >
> 
> Just to be clear, it won't be "external" library like OpenCV...
> 
> 
> > library, it will definitely not be used by FFmpeg codecs like you wish
> > in your first paragraph.
> >
> > If you want a fighting chance of a project that is not stillborn, I
> > think you need to make it part of FFmpeg, and make sure important
> >
> 
> .. it will be part of FFmpeg like libavfilter, just a new module -
> libmotion.
> 

yeah, but not a good idea. Just make it an optional component of
libavutil.

-- 
Clément B.


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [DISCUSSION] Motion Estimation API/Library

2017-08-01 Thread Davinder Singh 
Hi Nicolas,

On Tue, Aug 1, 2017 at 11:57 AM Nicolas George  wrote:

> Le quartidi 14 thermidor, an CCXXV, Davinder Singh a écrit :
> > As we've been planning since forever (
> > https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/snow.h#L182,
> > http://ffmpeg.org/pipermail/ffmpeg-devel/2016-July/197095.html) we need
> > Motion Estimation code that could be shared in codecs and motion filters.
> >
> > The idea is to make the Motion Estimation idependent of Encoders more
> > specifically - AVCodecContext.
>
> This is a very good idea.
>
> > So, I’ve moved motion estimation and me_cmp code to a new location -
> > libmotion . I
> > think it’s a good idea to make a new lib instead moving it to
> > libavutil (as discussed
> > previously <
> http://ffmpeg.org/pipermail/ffmpeg-devel/2016-July/197161.html>).
> > That way we can make it independent of everything else in FFmpeg.
>
> But this is not. Please no, not yet another library!
>
> A separate library like that will at the beginning only be used by the
> handful of hard-core developers. Unless it meets a wide success very
> fast, with very useful tools available immediately, it will soon be
> forgotten ("seems interesting, but not yet mature, I'll come back and
> see in six months") and start bitrotting as soon as you have moved to
> something else.
>
> In particular, the main policy of FFmpeg is to not depend on external
> libraries for core features. Therefore, if your project is a separate
>

Just to be clear, it won't be "external" library like OpenCV...


> library, it will definitely not be used by FFmpeg codecs like you wish
> in your first paragraph.
>
> If you want a fighting chance of a project that is not stillborn, I
> think you need to make it part of FFmpeg, and make sure important
>

.. it will be part of FFmpeg like libavfilter, just a new module -
libmotion.


> components of FFmpeg use it as soon as possible.
>
> Regards,
>
> --
>   Nicolas George
> [...]


Regards,
-- 
Davinder Singh
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [DISCUSSION] Motion Estimation API/Library

2017-08-01 Thread Nicolas George 
Le quartidi 14 thermidor, an CCXXV, Davinder Singh a écrit :
> As we've been planning since forever (
> https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/snow.h#L182,
> http://ffmpeg.org/pipermail/ffmpeg-devel/2016-July/197095.html) we need
> Motion Estimation code that could be shared in codecs and motion filters.
> 
> The idea is to make the Motion Estimation idependent of Encoders more
> specifically - AVCodecContext.

This is a very good idea.

> So, I’ve moved motion estimation and me_cmp code to a new location -
> libmotion . I
> think it’s a good idea to make a new lib instead moving it to
> libavutil (as discussed
> previously ).
> That way we can make it independent of everything else in FFmpeg.

But this is not. Please no, not yet another library!

A separate library like that will at the beginning only be used by the
handful of hard-core developers. Unless it meets a wide success very
fast, with very useful tools available immediately, it will soon be
forgotten ("seems interesting, but not yet mature, I'll come back and
see in six months") and start bitrotting as soon as you have moved to
something else.

In particular, the main policy of FFmpeg is to not depend on external
libraries for core features. Therefore, if your project is a separate
library, it will definitely not be used by FFmpeg codecs like you wish
in your first paragraph.

If you want a fighting chance of a project that is not stillborn, I
think you need to make it part of FFmpeg, and make sure important
components of FFmpeg use it as soon as possible.

Regards,

-- 
  Nicolas George


signature.asc
Description: Digital signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

[FFmpeg-devel] [DISCUSSION] Motion Estimation API/Library

2017-07-31 Thread Davinder Singh 
Hello everyone,

As we've been planning since forever (
https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/snow.h#L182,
http://ffmpeg.org/pipermail/ffmpeg-devel/2016-July/197095.html) we need
Motion Estimation code that could be shared in codecs and motion filters.

The idea is to make the Motion Estimation idependent of Encoders more
specifically - AVCodecContext.
So, I’ve moved motion estimation and me_cmp code to a new location -
libmotion . I
think it’s a good idea to make a new lib instead moving it to
libavutil (as discussed
previously ).
That way we can make it independent of everything else in FFmpeg.
Everything will be accessed through single context AVMotionEstContext, and
also be initialized through it. This API could also be used by researchers
to test new codecs or FRUC. And additionally, we can implement best Optical
Flow methods: http://vision.middlebury.edu/flow/eval/results/results-i1.php in
the future which are not available in OpenCV.

GOALS:
1. Make me_cmp_func()

independent of MpegEncContext and AVCodecContext. To accomplish this, we
can move following from MpegEncContext to MotionEstContext or[*]
MECmpContext (as these are used in cmp fxn in me_cmp.c
):
int mb_intra;
PixblockDSPContext pdsp;
FDCTDSPContext fdsp;
IDCTDSPContext idsp;
int (*fast_dct_quantize)();
void (*dct_unquantize_inter)();
void (*dct_unquantize_intra)();
int block_last_index[12];
int qscale;
ScanTable intra_scantable;
int ac_esc_length;
uint8_t *intra_ac_vlc_length;
uint8_t *intra_ac_vlc_last_length;
uint8_t *inter_ac_vlc_length;
uint8_t *inter_ac_vlc_last_length;
uint8_t *luma_dc_vlc_length;
MECmpContext mecc; [*]

As pointed by michaelni earlier, these are mostly more used by Encoders
than ME. If libavcodec is more appropriate place for some of these (e.g.
pdsp, fdsp) and these are also used by few cmp functions, like
FF_CMP_BIT, FF_CMP_RD, FF_CMP_DCT, FF_CMP_PSNR,
we can move these to specific encoders and set function address in Context
before using them. Or we can register cmp functions via by calling API
while initializing FFmpeg/Options. Only generic functions will be part of
libmotion.

2. Initialization of MECmpContext at one place (like init pdsp and dct if
moved in MECmpContext).
3. Make motion_est.c

code independent of MpegEncContext. for this a lot of things need to be
moved from MpegEncContext to MotionEstContext. MpegEncContext is used at
lot of places. Please suggest your approach on this. Generic ME should
surely be seperated.

Please share your views on this.

Regards,
-- 
Davinder Singh
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel