Re: [FFmpeg-devel] [PATCH] checkasm/af_afir: relax the max allowed absolute difference

2019-01-13 Thread Derek Buitenhuis
On 13/01/2019 18:24, James Almer wrote: > I thought i had amended the patch, but guess not... > > I can revert and reapply with the amended commit message if you want, > but it will kinda litter the tree for not a lot of gain. Then again, the > tree is already a mess with all the merges. Not

Re: [FFmpeg-devel] [PATCH] checkasm/af_afir: relax the max allowed absolute difference

2019-01-13 Thread James Almer
On 1/13/2019 3:18 PM, Derek Buitenhuis wrote: > On 13/01/2019 18:01, James Almer wrote: >> Pushes as is then. Thanks. > > Er. > > You didn't add the actual description of the problem/fix > to the commit message. I thought i had amended the patch, but guess not... I can revert and reapply with

Re: [FFmpeg-devel] [PATCH] checkasm/af_afir: relax the max allowed absolute difference

2019-01-13 Thread Derek Buitenhuis
On 13/01/2019 18:01, James Almer wrote: > Pushes as is then. Thanks. Er. You didn't add the actual description of the problem/fix to the commit message. - Derek ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org

Re: [FFmpeg-devel] [PATCH] checkasm/af_afir: relax the max allowed absolute difference

2019-01-13 Thread James Almer
On 1/13/2019 12:55 PM, Derek Buitenhuis wrote: > On 13/01/2019 15:45, James Almer wrote: >> If there is, i don't know it. >> >> Float based fate tests have been fine tuned before when different >> architectures proved a certain stddev value was not lax enough, so this >> one could be increased if

Re: [FFmpeg-devel] [PATCH] checkasm/af_afir: relax the max allowed absolute difference

2019-01-13 Thread Derek Buitenhuis
On 13/01/2019 15:45, James Almer wrote: > If there is, i don't know it. > > Float based fate tests have been fine tuned before when different > architectures proved a certain stddev value was not lax enough, so this > one could be increased if needed as well, but if you prefer i can use a > big

Re: [FFmpeg-devel] [PATCH] checkasm/af_afir: relax the max allowed absolute difference

2019-01-13 Thread James Almer
On 1/13/2019 11:32 AM, Derek Buitenhuis wrote: > On 13/01/2019 02:44, James Almer wrote: >> Michael suggested 1000*FLT_EPSILON but IMO that's too big and may hide >> errors in future implementations. >> The value i used is the smallest value i found that didn't fail after >> several runs. 6.1e-05

Re: [FFmpeg-devel] [PATCH] checkasm/af_afir: relax the max allowed absolute difference

2019-01-13 Thread Derek Buitenhuis
On 13/01/2019 02:44, James Almer wrote: > Michael suggested 1000*FLT_EPSILON but IMO that's too big and may hide > errors in future implementations. > The value i used is the smallest value i found that didn't fail after > several runs. 6.1e-05 for example fails. I figured that's how it was

Re: [FFmpeg-devel] [PATCH] checkasm/af_afir: relax the max allowed absolute difference

2019-01-12 Thread James Almer
On 1/11/2019 12:17 PM, Derek Buitenhuis wrote: > On 11/01/2019 13:28, Hendrik Leppkes wrote: >> Because the computation accumulates more inaccuarcy then FLT_EPSILON >> allows for. That value is really not of that great use. If you have >> two accurate numbers and do one calculation, it may work,

Re: [FFmpeg-devel] [PATCH] checkasm/af_afir: relax the max allowed absolute difference

2019-01-11 Thread Derek Buitenhuis
On 11/01/2019 13:28, Hendrik Leppkes wrote: > Because the computation accumulates more inaccuarcy then FLT_EPSILON > allows for. That value is really not of that great use. If you have > two accurate numbers and do one calculation, it may work, but if you > do a whole bunch of them, the error

Re: [FFmpeg-devel] [PATCH] checkasm/af_afir: relax the max allowed absolute difference

2019-01-11 Thread Hendrik Leppkes
On Fri, Jan 11, 2019 at 2:12 PM Derek Buitenhuis wrote: > > On 10/01/2019 23:34, James Almer wrote: > > -if (!float_near_abs_eps(cdst[i], odst[i], FLT_EPSILON)) { > > +if (!float_near_abs_eps(cdst[i], odst[i], 6.2e-05)) { > > Can you elaborate a bit more on this? FLT_EPSILON is

Re: [FFmpeg-devel] [PATCH] checkasm/af_afir: relax the max allowed absolute difference

2019-01-11 Thread Derek Buitenhuis
On 10/01/2019 23:34, James Almer wrote: > -if (!float_near_abs_eps(cdst[i], odst[i], FLT_EPSILON)) { > +if (!float_near_abs_eps(cdst[i], odst[i], 6.2e-05)) { Can you elaborate a bit more on this? FLT_EPSILON is used correctly here as far as I can tell, and it is not clear why it

[FFmpeg-devel] [PATCH] checkasm/af_afir: relax the max allowed absolute difference

2019-01-10 Thread James Almer
Should fix failures on x86_32. Signed-off-by: James Almer --- tests/checkasm/af_afir.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/checkasm/af_afir.c b/tests/checkasm/af_afir.c index 54e2f68d6c..e3fb76e8e0 100644 --- a/tests/checkasm/af_afir.c +++