Re: [FFmpeg-devel] [PATCH] avutil/crc: always use precalculated CRC tables for known polynomials

2017-10-25 Thread Muhammad Faiz
On Tue, Oct 24, 2017 at 6:05 PM, Michael Niedermayer
 wrote:
> On Mon, Oct 23, 2017 at 10:36:21PM -0300, James Almer wrote:
>> On 10/23/2017 10:24 PM, Michael Niedermayer wrote:
>> > On Tue, Oct 24, 2017 at 01:39:03AM +0100, Mark Thompson wrote:
>> >> On 24/10/17 00:52, Michael Niedermayer wrote:
>> >>> Hi
>> >>>
>> >>> On Mon, Oct 23, 2017 at 04:43:19PM +0100, Mark Thompson wrote:
>>  On 23/10/17 03:56, Michael Niedermayer wrote:
>> > the initialization should be thread safe as it never writes a different
>> > value in the same spot
>> 
>>  This is not true; please be very careful with assumptions like this.
>> 
>>  The C standard calls this a data race and it is undefined behaviour.
>> >>>
>> >>> Thats interresting, we definitly do not want undefined behavior
>> >>> can you quote the relevant parts of the C standard
>> >>> which make writing the same value, a data race ?
>> >>>
>> >>> I was not aware of this if its true.
>> >>> Also i dont immedeatly see this in the "latest" draft i am looking at
>> >>> atm
>> >>>
>> >>> What i see is this:
>> >>> 4 Two expression evaluations conflict if one of them modifies a memory 
>> >>> location and the
>> >>>   other one reads or modifies the same memory location.
>> >>> ...
>> >>> 25 The execution of a program contains a data race if it contains two 
>> >>> conflicting actions in
>> >>>different threads, at least one of which is not atomic, and neither 
>> >>> happens before the
>> >>>other. Any such data race results in undefined behavior.
>> >>>
>> >>> This seems carefully worded to not include writing the same value.
>> >>> As "modification" implies change which does not occur when writing the
>> >>> same value.
>> >>
>> >> All writes are modifications.
>> >>
>> >> See section 3.1 note 2:
>> >> """
>> >> 3 NOTE 2   "Modify"€™ includes the case where the new value being stored 
>> >> is the same as the previous value.
>> >> """
>> >
>> > That resolves the difference between our interpretation
>> >
>> > thanks
>>
>> Should i push this, or does someone want to give ff_thread_once() a try?
>
> If ff_thread_once() is used outside init code, it should be benchmarked
> as it may be a bad idea speed wise.
> During init speed doesnt matter so ff_thread_once() there would be fine
> but would require more changes to move it to init.

Using my patch:
Benchmark: make fate-lavf-png at pngenc.c:png_write_chunk
old:
 150640 decicycles in av_crc_get_table:first, 1 runs,  0 skips
   1740 decicycles in av_crc_get_table,   1 runs,  0 skips
   1390 decicycles in av_crc_get_table,   2 runs,  0 skips
945 decicycles in av_crc_get_table,   4 runs,  0 skips
720 decicycles in av_crc_get_table,   8 runs,  0 skips
592 decicycles in av_crc_get_table,  16 runs,  0 skips
536 decicycles in av_crc_get_table,  32 runs,  0 skips
510 decicycles in av_crc_get_table,  64 runs,  0 skips
497 decicycles in av_crc_get_table, 128 runs,  0 skips
494 decicycles in av_crc_get_table, 256 runs,  0 skips
479 decicycles in av_crc_get_table, 512 runs,  0 skips
454 decicycles in av_crc_get_table,1024 runs,  0 skips
434 decicycles in av_crc_get_table,2048 runs,  0 skips


new:
 235060 decicycles in av_crc_get_table:first, 1 runs,  0 skips
   2900 decicycles in av_crc_get_table,   1 runs,  0 skips
   2560 decicycles in av_crc_get_table,   2 runs,  0 skips
   1825 decicycles in av_crc_get_table,   4 runs,  0 skips
   1200 decicycles in av_crc_get_table,   8 runs,  0 skips
902 decicycles in av_crc_get_table,  16 runs,  0 skips
719 decicycles in av_crc_get_table,  32 runs,  0 skips
628 decicycles in av_crc_get_table,  64 runs,  0 skips
576 decicycles in av_crc_get_table, 128 runs,  0 skips
550 decicycles in av_crc_get_table, 256 runs,  0 skips
532 decicycles in av_crc_get_table, 512 runs,  0 skips
499 decicycles in av_crc_get_table,1024 runs,  0 skips
470 decicycles in av_crc_get_table,2048 runs,  0 skips

Thank's.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


Re: [FFmpeg-devel] [PATCH] avutil/crc: always use precalculated CRC tables for known polynomials

2017-10-24 Thread Michael Niedermayer
On Mon, Oct 23, 2017 at 10:36:21PM -0300, James Almer wrote:
> On 10/23/2017 10:24 PM, Michael Niedermayer wrote:
> > On Tue, Oct 24, 2017 at 01:39:03AM +0100, Mark Thompson wrote:
> >> On 24/10/17 00:52, Michael Niedermayer wrote:
> >>> Hi
> >>>
> >>> On Mon, Oct 23, 2017 at 04:43:19PM +0100, Mark Thompson wrote:
>  On 23/10/17 03:56, Michael Niedermayer wrote:
> > the initialization should be thread safe as it never writes a different
> > value in the same spot
> 
>  This is not true; please be very careful with assumptions like this.
> 
>  The C standard calls this a data race and it is undefined behaviour.
> >>>
> >>> Thats interresting, we definitly do not want undefined behavior
> >>> can you quote the relevant parts of the C standard
> >>> which make writing the same value, a data race ?
> >>>
> >>> I was not aware of this if its true.
> >>> Also i dont immedeatly see this in the "latest" draft i am looking at
> >>> atm
> >>>
> >>> What i see is this:
> >>> 4 Two expression evaluations conflict if one of them modifies a memory 
> >>> location and the
> >>>   other one reads or modifies the same memory location.
> >>> ...
> >>> 25 The execution of a program contains a data race if it contains two 
> >>> conflicting actions in
> >>>different threads, at least one of which is not atomic, and neither 
> >>> happens before the
> >>>other. Any such data race results in undefined behavior.
> >>>
> >>> This seems carefully worded to not include writing the same value.
> >>> As "modification" implies change which does not occur when writing the
> >>> same value.
> >>
> >> All writes are modifications.
> >>
> >> See section 3.1 note 2:
> >> """
> >> 3 NOTE 2   "Modify"€™ includes the case where the new value being stored 
> >> is the same as the previous value.
> >> """
> > 
> > That resolves the difference between our interpretation
> > 
> > thanks
> 
> Should i push this, or does someone want to give ff_thread_once() a try?

If ff_thread_once() is used outside init code, it should be benchmarked
as it may be a bad idea speed wise.
During init speed doesnt matter so ff_thread_once() there would be fine
but would require more changes to move it to init.

I have no oppinion about ff_thread_once() vs hardcoding ATM

thx

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The greatest way to live with honor in this world is to be what we pretend
to be. -- Socrates


signature.asc
Description: Digital signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


Re: [FFmpeg-devel] [PATCH] avutil/crc: always use precalculated CRC tables for known polynomials

2017-10-24 Thread Muhammad Faiz
On Tue, Oct 24, 2017 at 8:36 AM, James Almer  wrote:
> On 10/23/2017 10:24 PM, Michael Niedermayer wrote:
>> On Tue, Oct 24, 2017 at 01:39:03AM +0100, Mark Thompson wrote:
>>> On 24/10/17 00:52, Michael Niedermayer wrote:
 Hi

 On Mon, Oct 23, 2017 at 04:43:19PM +0100, Mark Thompson wrote:
> On 23/10/17 03:56, Michael Niedermayer wrote:
>> the initialization should be thread safe as it never writes a different
>> value in the same spot
>
> This is not true; please be very careful with assumptions like this.
>
> The C standard calls this a data race and it is undefined behaviour.

 Thats interresting, we definitly do not want undefined behavior
 can you quote the relevant parts of the C standard
 which make writing the same value, a data race ?

 I was not aware of this if its true.
 Also i dont immedeatly see this in the "latest" draft i am looking at
 atm

 What i see is this:
 4 Two expression evaluations conflict if one of them modifies a memory 
 location and the
   other one reads or modifies the same memory location.
 ...
 25 The execution of a program contains a data race if it contains two 
 conflicting actions in
different threads, at least one of which is not atomic, and neither 
 happens before the
other. Any such data race results in undefined behavior.

 This seems carefully worded to not include writing the same value.
 As "modification" implies change which does not occur when writing the
 same value.
>>>
>>> All writes are modifications.
>>>
>>> See section 3.1 note 2:
>>> """
>>> 3 NOTE 2   "Modify"€™ includes the case where the new value being stored is 
>>> the same as the previous value.
>>> """
>>
>> That resolves the difference between our interpretation
>>
>> thanks
>
> Should i push this, or does someone want to give ff_thread_once() a try?

I've posted a patch.

Thank's.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


Re: [FFmpeg-devel] [PATCH] avutil/crc: always use precalculated CRC tables for known polynomials

2017-10-23 Thread James Almer
On 10/23/2017 10:24 PM, Michael Niedermayer wrote:
> On Tue, Oct 24, 2017 at 01:39:03AM +0100, Mark Thompson wrote:
>> On 24/10/17 00:52, Michael Niedermayer wrote:
>>> Hi
>>>
>>> On Mon, Oct 23, 2017 at 04:43:19PM +0100, Mark Thompson wrote:
 On 23/10/17 03:56, Michael Niedermayer wrote:
> the initialization should be thread safe as it never writes a different
> value in the same spot

 This is not true; please be very careful with assumptions like this.

 The C standard calls this a data race and it is undefined behaviour.
>>>
>>> Thats interresting, we definitly do not want undefined behavior
>>> can you quote the relevant parts of the C standard
>>> which make writing the same value, a data race ?
>>>
>>> I was not aware of this if its true.
>>> Also i dont immedeatly see this in the "latest" draft i am looking at
>>> atm
>>>
>>> What i see is this:
>>> 4 Two expression evaluations conflict if one of them modifies a memory 
>>> location and the
>>>   other one reads or modifies the same memory location.
>>> ...
>>> 25 The execution of a program contains a data race if it contains two 
>>> conflicting actions in
>>>different threads, at least one of which is not atomic, and neither 
>>> happens before the
>>>other. Any such data race results in undefined behavior.
>>>
>>> This seems carefully worded to not include writing the same value.
>>> As "modification" implies change which does not occur when writing the
>>> same value.
>>
>> All writes are modifications.
>>
>> See section 3.1 note 2:
>> """
>> 3 NOTE 2   "Modify"€™ includes the case where the new value being stored is 
>> the same as the previous value.
>> """
> 
> That resolves the difference between our interpretation
> 
> thanks

Should i push this, or does someone want to give ff_thread_once() a try?
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


Re: [FFmpeg-devel] [PATCH] avutil/crc: always use precalculated CRC tables for known polynomials

2017-10-23 Thread Michael Niedermayer
On Tue, Oct 24, 2017 at 01:39:03AM +0100, Mark Thompson wrote:
> On 24/10/17 00:52, Michael Niedermayer wrote:
> > Hi
> > 
> > On Mon, Oct 23, 2017 at 04:43:19PM +0100, Mark Thompson wrote:
> >> On 23/10/17 03:56, Michael Niedermayer wrote:
> >>> the initialization should be thread safe as it never writes a different
> >>> value in the same spot
> >>
> >> This is not true; please be very careful with assumptions like this.
> >>
> >> The C standard calls this a data race and it is undefined behaviour.
> > 
> > Thats interresting, we definitly do not want undefined behavior
> > can you quote the relevant parts of the C standard
> > which make writing the same value, a data race ?
> > 
> > I was not aware of this if its true.
> > Also i dont immedeatly see this in the "latest" draft i am looking at
> > atm
> > 
> > What i see is this:
> > 4 Two expression evaluations conflict if one of them modifies a memory 
> > location and the
> >   other one reads or modifies the same memory location.
> > ...
> > 25 The execution of a program contains a data race if it contains two 
> > conflicting actions in
> >different threads, at least one of which is not atomic, and neither 
> > happens before the
> >other. Any such data race results in undefined behavior.
> > 
> > This seems carefully worded to not include writing the same value.
> > As "modification" implies change which does not occur when writing the
> > same value.
> 
> All writes are modifications.
> 
> See section 3.1 note 2:
> """
> 3 NOTE 2   "Modify"€™ includes the case where the new value being stored is 
> the same as the previous value.
> """

That resolves the difference between our interpretation

thanks

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Elect your leaders based on what they did after the last election, not
based on what they say before an election.



signature.asc
Description: Digital signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


Re: [FFmpeg-devel] [PATCH] avutil/crc: always use precalculated CRC tables for known polynomials

2017-10-23 Thread Mark Thompson
On 24/10/17 00:52, Michael Niedermayer wrote:
> Hi
> 
> On Mon, Oct 23, 2017 at 04:43:19PM +0100, Mark Thompson wrote:
>> On 23/10/17 03:56, Michael Niedermayer wrote:
>>> the initialization should be thread safe as it never writes a different
>>> value in the same spot
>>
>> This is not true; please be very careful with assumptions like this.
>>
>> The C standard calls this a data race and it is undefined behaviour.
> 
> Thats interresting, we definitly do not want undefined behavior
> can you quote the relevant parts of the C standard
> which make writing the same value, a data race ?
> 
> I was not aware of this if its true.
> Also i dont immedeatly see this in the "latest" draft i am looking at
> atm
> 
> What i see is this:
> 4 Two expression evaluations conflict if one of them modifies a memory 
> location and the
>   other one reads or modifies the same memory location.
> ...
> 25 The execution of a program contains a data race if it contains two 
> conflicting actions in
>different threads, at least one of which is not atomic, and neither 
> happens before the
>other. Any such data race results in undefined behavior.
> 
> This seems carefully worded to not include writing the same value.
> As "modification" implies change which does not occur when writing the
> same value.

All writes are modifications.

See section 3.1 note 2:
"""
3 NOTE 2   "Modify"€™ includes the case where the new value being stored is the 
same as the previous value.
"""

>> It is not just a theoretical concern, either - on architectures with 
>> destructive write-hint instructions ("fill cache line with unspecified data 
>> without loading it from memory, because I'm about to overwrite all of it", 
>> exactly what you want to use (and therefore the compiler will generate) to 
>> avoid pointless loads when overwriting a large table) other threads can and 
>> do see different contents transiently when the same data is written to the 
>> location.
> 
> This might violate:
> 27 NOTE 13 Compiler transformations that introduce assignments to a 
> potentially shared memory location
>that would not be modified by the abstract machine are generally precluded 
> by this standard, since such an
>assignment might overwrite another assignment by a different thread in 
> cases in which an abstract machine
>execution would not have encountered a data race. This includes 
> implementations of data member
>assignment that overwrite adjacent members in separate memory locations. 
> We also generally preclude
>reordering of atomic loads in cases in which the atomics in question may 
> alias, since this may violate the
>"visible sequence" rules.

Yes, in order to match the behaviour of the abstract machine the transformation 
requires that the region in question is:
* Not atomic: if atomic, another thread would be allowed to observe the 
intermediate state directly.
* Overwritten entirely: if not overwritten entirely, other data would be 
touched which should not be, and that could be observed by another thread.
* Without external ordering: if external synchronisation were present, another 
thread would be able to read values on one or other side of that boundary 
without a data race and observe an inconsistent state.

The CRC table construction satisfies all of these conditions.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


Re: [FFmpeg-devel] [PATCH] avutil/crc: always use precalculated CRC tables for known polynomials

2017-10-23 Thread Michael Niedermayer
Hi

On Mon, Oct 23, 2017 at 04:43:19PM +0100, Mark Thompson wrote:
> On 23/10/17 03:56, Michael Niedermayer wrote:
> > the initialization should be thread safe as it never writes a different
> > value in the same spot
> 
> This is not true; please be very careful with assumptions like this.
> 
> The C standard calls this a data race and it is undefined behaviour.

Thats interresting, we definitly do not want undefined behavior
can you quote the relevant parts of the C standard
which make writing the same value, a data race ?

I was not aware of this if its true.
Also i dont immedeatly see this in the "latest" draft i am looking at
atm

What i see is this:
4 Two expression evaluations conflict if one of them modifies a memory location 
and the
  other one reads or modifies the same memory location.
...
25 The execution of a program contains a data race if it contains two 
conflicting actions in
   different threads, at least one of which is not atomic, and neither happens 
before the
   other. Any such data race results in undefined behavior.

This seems carefully worded to not include writing the same value.
As "modification" implies change which does not occur when writing the
same value.


> 
> It is not just a theoretical concern, either - on architectures with 
> destructive write-hint instructions ("fill cache line with unspecified data 
> without loading it from memory, because I'm about to overwrite all of it", 
> exactly what you want to use (and therefore the compiler will generate) to 
> avoid pointless loads when overwriting a large table) other threads can and 
> do see different contents transiently when the same data is written to the 
> location.

This might violate:
27 NOTE 13 Compiler transformations that introduce assignments to a potentially 
shared memory location
   that would not be modified by the abstract machine are generally precluded 
by this standard, since such an
   assignment might overwrite another assignment by a different thread in cases 
in which an abstract machine
   execution would not have encountered a data race. This includes 
implementations of data member
   assignment that overwrite adjacent members in separate memory locations. We 
also generally preclude
   reordering of atomic loads in cases in which the atomics in question may 
alias, since this may violate the
   "visible sequence" rules.

[...]

-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

"You are 36 times more likely to die in a bathtub than at the hands of a
terrorist. Also, you are 2.5 times more likely to become a president and
2 times more likely to become an astronaut, than to die in a terrorist
attack." -- Thoughty2



signature.asc
Description: Digital signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


Re: [FFmpeg-devel] [PATCH] avutil/crc: always use precalculated CRC tables for known polynomials

2017-10-23 Thread Mark Thompson
On 23/10/17 03:56, Michael Niedermayer wrote:
> the initialization should be thread safe as it never writes a different
> value in the same spot

This is not true; please be very careful with assumptions like this.

The C standard calls this a data race and it is undefined behaviour.

It is not just a theoretical concern, either - on architectures with 
destructive write-hint instructions ("fill cache line with unspecified data 
without loading it from memory, because I'm about to overwrite all of it", 
exactly what you want to use (and therefore the compiler will generate) to 
avoid pointless loads when overwriting a large table) other threads can and do 
see different contents transiently when the same data is written to the 
location.

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


Re: [FFmpeg-devel] [PATCH] avutil/crc: always use precalculated CRC tables for known polynomials

2017-10-23 Thread Derek Buitenhuis
On 10/23/2017 3:56 AM, Michael Niedermayer wrote:
> the initialization should be thread safe as it never writes a different
> value in the same spot
> This would avoid the need to alwas hardcode the tables

Still no reason to *not* put it under ff_thread_once, though,
to minimize work done.

- Derek
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


Re: [FFmpeg-devel] [PATCH] avutil/crc: always use precalculated CRC tables for known polynomials

2017-10-22 Thread Michael Niedermayer
On Sun, Oct 22, 2017 at 10:03:55AM -0300, James Almer wrote:
> This prevents data races in av_crc_get_table()

the modules which use a specific crc can init it during their
initialization.
the initialization should be thread safe as it never writes a different
value in the same spot
This would avoid the need to alwas hardcode the tables

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The misfortune of the wise is better than the prosperity of the fool.
-- Epicurus


signature.asc
Description: Digital signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


Re: [FFmpeg-devel] [PATCH] avutil/crc: always use precalculated CRC tables for known polynomials

2017-10-22 Thread Carl Eugen Hoyos
2017-10-22 15:03 GMT+02:00 James Almer :
> This prevents data races in av_crc_get_table()

Doesn't this mix two changes - moving the tables into a
dedicated file and making them static - that are not
necessarily related?

Carl Eugen
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


Re: [FFmpeg-devel] [PATCH] avutil/crc: always use precalculated CRC tables for known polynomials

2017-10-22 Thread Derek Buitenhuis
On 10/22/2017 2:11 PM, James Almer wrote:
> It was suggested, but nobody gave it a try (Or they did but found it
> wasn't as simple as first thought?).

[...]

> Thread sanitizer complains about this in every other run, and the tables
> are at most 1k each, so this is not a bad solution and can be replaced
> by ff_thread_once in the future.

And say that N times and you end up with Nk of tables ;).

That said, I'm not NAKing the patch, just suggesting an alternative.

- Derek
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


Re: [FFmpeg-devel] [PATCH] avutil/crc: always use precalculated CRC tables for known polynomials

2017-10-22 Thread James Almer
On 10/22/2017 10:08 AM, Derek Buitenhuis wrote:
> On 10/22/2017 2:03 PM, James Almer wrote:
>> This prevents data races in av_crc_get_table()
>>
>> Signed-off-by: James Almer 
>> ---
>>  libavutil/Makefile |1 +
>>  libavutil/crc.c|  295 +-
>>  libavutil/crc_tables.c | 1030 
>> 
>>  libavutil/crc_tables.h |   33 ++
>>  4 files changed, 1066 insertions(+), 293 deletions(-)
>>  create mode 100644 libavutil/crc_tables.c
>>  create mode 100644 libavutil/crc_tables.h
> 
> Can this be generated at init, or lazily, using ff_thread_once instead of
> hardcoding huge tables?
> 
> - Derek

It was suggested, but nobody gave it a try (Or they did but found it
wasn't as simple as first thought?).

Thread sanitizer complains about this in every other run, and the tables
are at most 1k each, so this is not a bad solution and can be replaced
by ff_thread_once in the future.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


Re: [FFmpeg-devel] [PATCH] avutil/crc: always use precalculated CRC tables for known polynomials

2017-10-22 Thread Derek Buitenhuis
On 10/22/2017 2:03 PM, James Almer wrote:
> This prevents data races in av_crc_get_table()
> 
> Signed-off-by: James Almer 
> ---
>  libavutil/Makefile |1 +
>  libavutil/crc.c|  295 +-
>  libavutil/crc_tables.c | 1030 
> 
>  libavutil/crc_tables.h |   33 ++
>  4 files changed, 1066 insertions(+), 293 deletions(-)
>  create mode 100644 libavutil/crc_tables.c
>  create mode 100644 libavutil/crc_tables.h

Can this be generated at init, or lazily, using ff_thread_once instead of
hardcoding huge tables?

- Derek
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


[FFmpeg-devel] [PATCH] avutil/crc: always use precalculated CRC tables for known polynomials

2017-10-22 Thread James Almer
This prevents data races in av_crc_get_table()

Signed-off-by: James Almer 
---
 libavutil/Makefile |1 +
 libavutil/crc.c|  295 +-
 libavutil/crc_tables.c | 1030 
 libavutil/crc_tables.h |   33 ++
 4 files changed, 1066 insertions(+), 293 deletions(-)
 create mode 100644 libavutil/crc_tables.c
 create mode 100644 libavutil/crc_tables.h

diff --git a/libavutil/Makefile b/libavutil/Makefile
index 4fe81fdd07..88a5c725c1 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -102,6 +102,7 @@ OBJS = adler32.o
\
color_utils.o\
cpu.o\
crc.o\
+   crc_tables.o \
des.o\
dict.o   \
display.o\
diff --git a/libavutil/crc.c b/libavutil/crc.c
index 495732b163..ed94368f68 100644
--- a/libavutil/crc.c
+++ b/libavutil/crc.c
@@ -23,289 +23,7 @@
 #include "bswap.h"
 #include "common.h"
 #include "crc.h"
-
-#if CONFIG_HARDCODED_TABLES
-static const AVCRC av_crc_table[AV_CRC_MAX][257] = {
-[AV_CRC_8_ATM] = {
-0x00, 0x07, 0x0E, 0x09, 0x1C, 0x1B, 0x12, 0x15, 0x38, 0x3F, 0x36, 0x31,
-0x24, 0x23, 0x2A, 0x2D, 0x70, 0x77, 0x7E, 0x79, 0x6C, 0x6B, 0x62, 0x65,
-0x48, 0x4F, 0x46, 0x41, 0x54, 0x53, 0x5A, 0x5D, 0xE0, 0xE7, 0xEE, 0xE9,
-0xFC, 0xFB, 0xF2, 0xF5, 0xD8, 0xDF, 0xD6, 0xD1, 0xC4, 0xC3, 0xCA, 0xCD,
-0x90, 0x97, 0x9E, 0x99, 0x8C, 0x8B, 0x82, 0x85, 0xA8, 0xAF, 0xA6, 0xA1,
-0xB4, 0xB3, 0xBA, 0xBD, 0xC7, 0xC0, 0xC9, 0xCE, 0xDB, 0xDC, 0xD5, 0xD2,
-0xFF, 0xF8, 0xF1, 0xF6, 0xE3, 0xE4, 0xED, 0xEA, 0xB7, 0xB0, 0xB9, 0xBE,
-0xAB, 0xAC, 0xA5, 0xA2, 0x8F, 0x88, 0x81, 0x86, 0x93, 0x94, 0x9D, 0x9A,
-0x27, 0x20, 0x29, 0x2E, 0x3B, 0x3C, 0x35, 0x32, 0x1F, 0x18, 0x11, 0x16,
-0x03, 0x04, 0x0D, 0x0A, 0x57, 0x50, 0x59, 0x5E, 0x4B, 0x4C, 0x45, 0x42,
-0x6F, 0x68, 0x61, 0x66, 0x73, 0x74, 0x7D, 0x7A, 0x89, 0x8E, 0x87, 0x80,
-0x95, 0x92, 0x9B, 0x9C, 0xB1, 0xB6, 0xBF, 0xB8, 0xAD, 0xAA, 0xA3, 0xA4,
-0xF9, 0xFE, 0xF7, 0xF0, 0xE5, 0xE2, 0xEB, 0xEC, 0xC1, 0xC6, 0xCF, 0xC8,
-0xDD, 0xDA, 0xD3, 0xD4, 0x69, 0x6E, 0x67, 0x60, 0x75, 0x72, 0x7B, 0x7C,
-0x51, 0x56, 0x5F, 0x58, 0x4D, 0x4A, 0x43, 0x44, 0x19, 0x1E, 0x17, 0x10,
-0x05, 0x02, 0x0B, 0x0C, 0x21, 0x26, 0x2F, 0x28, 0x3D, 0x3A, 0x33, 0x34,
-0x4E, 0x49, 0x40, 0x47, 0x52, 0x55, 0x5C, 0x5B, 0x76, 0x71, 0x78, 0x7F,
-0x6A, 0x6D, 0x64, 0x63, 0x3E, 0x39, 0x30, 0x37, 0x22, 0x25, 0x2C, 0x2B,
-0x06, 0x01, 0x08, 0x0F, 0x1A, 0x1D, 0x14, 0x13, 0xAE, 0xA9, 0xA0, 0xA7,
-0xB2, 0xB5, 0xBC, 0xBB, 0x96, 0x91, 0x98, 0x9F, 0x8A, 0x8D, 0x84, 0x83,
-0xDE, 0xD9, 0xD0, 0xD7, 0xC2, 0xC5, 0xCC, 0xCB, 0xE6, 0xE1, 0xE8, 0xEF,
-0xFA, 0xFD, 0xF4, 0xF3, 0x01
-},
-[AV_CRC_16_ANSI] = {
-0x, 0x0580, 0x0F80, 0x0A00, 0x1B80, 0x1E00, 0x1400, 0x1180,
-0x3380, 0x3600, 0x3C00, 0x3980, 0x2800, 0x2D80, 0x2780, 0x2200,
-0x6380, 0x6600, 0x6C00, 0x6980, 0x7800, 0x7D80, 0x7780, 0x7200,
-0x5000, 0x5580, 0x5F80, 0x5A00, 0x4B80, 0x4E00, 0x4400, 0x4180,
-0xC380, 0xC600, 0xCC00, 0xC980, 0xD800, 0xDD80, 0xD780, 0xD200,
-0xF000, 0xF580, 0xFF80, 0xFA00, 0xEB80, 0xEE00, 0xE400, 0xE180,
-0xA000, 0xA580, 0xAF80, 0xAA00, 0xBB80, 0xBE00, 0xB400, 0xB180,
-0x9380, 0x9600, 0x9C00, 0x9980, 0x8800, 0x8D80, 0x8780, 0x8200,
-0x8381, 0x8601, 0x8C01, 0x8981, 0x9801, 0x9D81, 0x9781, 0x9201,
-0xB001, 0xB581, 0xBF81, 0xBA01, 0xAB81, 0xAE01, 0xA401, 0xA181,
-0xE001, 0xE581, 0xEF81, 0xEA01, 0xFB81, 0xFE01, 0xF401, 0xF181,
-0xD381, 0xD601, 0xDC01, 0xD981, 0xC801, 0xCD81, 0xC781, 0xC201,
-0x4001, 0x4581, 0x4F81, 0x4A01, 0x5B81, 0x5E01, 0x5401, 0x5181,
-0x7381, 0x7601, 0x7C01, 0x7981, 0x6801, 0x6D81, 0x6781, 0x6201,
-0x2381, 0x2601, 0x2C01, 0x2981, 0x3801, 0x3D81, 0x3781, 0x3201,
-0x1001, 0x1581, 0x1F81, 0x1A01, 0x0B81, 0x0E01, 0x0401, 0x0181,
-0x0383, 0x0603, 0x0C03, 0x0983, 0x1803, 0x1D83, 0x1783, 0x1203,
-0x3003, 0x3583, 0x3F83, 0x3A03, 0x2B83, 0x2E03, 0x2403, 0x2183,
-0x6003, 0x6583, 0x6F83, 0x6A03, 0x7B83, 0x7E03, 0x7403, 0x7183,
-0x5383, 0x5603, 0x5C03, 0x5983, 0x4803, 0x4D83, 0x4783, 0x4203,
-0xC003, 0xC583, 0xCF83, 0xCA03, 0xDB83, 0xDE03, 0xD403, 0xD183,
-0xF383, 0xF603, 0xFC03, 0xF983, 0xE803, 0xED83, 0xE783, 0xE203,
-0xA383, 0xA603, 0xAC03, 0xA983, 0xB803, 0xBD83, 0xB783, 0xB203,
-0x9003, 0x9583, 0x9F83, 0x9A03, 0x8B83, 0x8E03, 0