[issue26040] Improve coverage and rigour of test.test_math

[Mark Dickinson]

Mark Dickinson added the comment:

Opened #27953 to track the issue with tan on OS X Tiger. Re-closing here.

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Mark Dickinson]

Mark Dickinson added the comment:

> or at least, no more unhappy than before

Not quite true: test_math and test_cmath are failing for the x86 Tiger 3.x 
buildbot:

==
FAIL: test_testfile (test.test_math.MathTests)
--
Traceback (most recent call last):
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/test_math.py", 
line 1190, in test_testfile
'\n  '.join(failures))
AssertionError: Failures in test_testfile:
  tan0064: tan(1.5707963267948961): expected 1978937966095219.0, got 
1978945885716843.0 (error = 7.92e+09 (31678486496 ulps); permitted error = 0 or 
5 ulps)


==
FAIL: test_specific_values (test.test_cmath.CMathTests)
--
Traceback (most recent call last):
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/test_cmath.py", 
line 398, in test_specific_values
msg=error_message)
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/test_cmath.py", 
line 147, in rAssertAlmostEqual
'{!r} and {!r} are not sufficiently close'.format(a, b))
AssertionError: tan0064: tan(complex(1.5707963267948961, 0.0))
Expected: complex(1978937966095219.0, 0.0)
Received: complex(1978945885716843.0, 0.0)
Received value insufficiently close to expected value.


It's probably not worth trying to fix this for such an ancient version of OS X. 
Re-opening; I'll aim to skip the test on the platform.

--
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Mark Dickinson]

Mark Dickinson added the comment:

Buildbots seem happy (or at least, no more unhappy than before). Closing.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Mark Dickinson]

Mark Dickinson added the comment:

iss26040_v4.patch applied. I'm watching the buildbots; if everything looks good 
there, I'll close the issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Roundup Robot]

Roundup Robot added the comment:

New changeset e3dbe8b7279a by Mark Dickinson in branch 'default':
Issue #26040: Improve test_math and test_cmath coverage and rigour. Thanks Jeff 
Allen.
https://hg.python.org/cpython/rev/e3dbe8b7279a

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Jeff Allen]

Jeff Allen added the comment:

Ah, cunning: I can make sense of it in hex.

>>> hex(to_ulps(expected))
'0x3ff0'
>>> hex(to_ulps(got))
'0x3fec'
>>> hex( to_ulps(got) - to_ulps(expected) )
'-0x4'

... and what you've done with ulp then follows.

In my version a format like "{:d} ulps" was a bad idea when the error was a 
gross one, but your to_ulps is only piece-wise linear -- large differences are 
compressed.

I'm pleased my work has mostly survived: here's hoping the house build-bots 
agree. erfc() is perhaps the last worry, but math & cmath  pass on my machine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Jeff Allen]

Jeff Allen added the comment:

Mark: Thanks for doing my homework. Points 1 and 3 I can readily agree with. I 
must take another look at to_ulps() with your patch on locally. I used the 
approach I did because I thought it was incorrect in exactly those corners 
where you prefer it. I'll take a closer look.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Mark Dickinson]

Mark Dickinson added the comment:

One more version; increased the default ulp_tol to 5 everywhere, and made some 
minor style fixes in nearby code.

--
Added file: https://bugs.python.org/file44254/iss26040_v4.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Mark Dickinson]

Mark Dickinson added the comment:

I've reviewed the test_math portion of the patch (which looks good, thank 
you!), and made a few revisions in iss26040_v3.patch.

1. I've rewritten the ulp() function to use struct. Same behaviour as before, 
but this way feels safer, since it doesn't rely on the exact form of the 
float.hex format.

2. I've reworked the ulp-based comparison to use the old to_ulps function, 
instead of basing it on ulp(). That's mostly just a personal preference: I 
prefer the symmetry of the abs(to_ulps(got) - to_ulps(expected)) form, and I 
think it behaves slightly better at or near powers of two. (If expected == 1.0 
and got == 1.0 - 2**-51, I want that to be seen as 4 ulps error rather than 2.)

3. I increased the default ulp tolerance to 5 ulps, and increased the ulps 
tolerance for some of the special cases. There are some badly behaved libm 
implementations out there, and I don't want to break more buildbots than 
necessary.

--
Added file: https://bugs.python.org/file44253/iss26040_v3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Mark Dickinson]

Mark Dickinson added the comment:

iss26040.patch didn't apply cleanly to master (because of the recent math.tau 
addition). Here's an updated patch.

--
Added file: https://bugs.python.org/file44251/iss26040_v2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Jeff Allen]

Jeff Allen added the comment:

Mark: Thanks for validating the additional cases so carefully.

If you still want to apply it in stages then I suppose the change to the 
comparison logic could go first (untested idea), although that's also where I 
could most easily have made a mistake.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Mark Dickinson]

Mark Dickinson added the comment:

Reverted in https://hg.python.org/cpython/rev/4389aa3507c5

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Mark Dickinson]

Mark Dickinson added the comment:

Thanks, Ned. (And apologies.) The test_math tests are too strict in this case, 
which is one of the issues that Jeff's full patch fixes. It looks like my 
strategy of applying the patch in two pieces isn't going to work.

I'll revert the change for now, and have another go at this tomorrow.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Ned Deily]

Ned Deily added the comment:

Mark, many buildbots are unhappy.  For example:

==
FAIL: test_testfile (test.test_math.MathTests)
--
Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_math.py", 
line 1106, in test_testfile
self.ftest("%s:%s(%r)" % (id, fn, ar), result, er)
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_math.py", 
line 194, in ftest
(name, value, expected))
AssertionError: cosh0063:cosh(709.7827) returned 8.988349783319008e+307, 
expected 8.988349783319007e+307

--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[koobs]

Changes by koobs :


--
nosy: +koobs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Roundup Robot]

Roundup Robot added the comment:

New changeset 1017215f5492 by Mark Dickinson in branch 'default':
Issue #26040 (part 1): add new testcases to cmath_testcases.txt. Thanks Jeff 
Allen.
https://hg.python.org/cpython/rev/1017215f5492

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Mark Dickinson]

Mark Dickinson added the comment:

I finally have some time to look at this. I've double-checked all the new cmath 
testcases against MPFR (via bigfloat), and they all look good. I plan to apply 
this in two stages: (1) apply the new cmath testcases, (2) apply the test_math 
changes, and (1.5) watch for buildbot failures in-between.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Mark Dickinson]

Changes by Mark Dickinson :


--
assignee:  -> mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Jeff Allen]

Changes by Jeff Allen :


Added file: http://bugs.python.org/file42191/extra_cmath_testcases.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Jeff Allen]

Changes by Jeff Allen :


Removed file: http://bugs.python.org/file42190/stat_math.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Jeff Allen]

Changes by Jeff Allen :


Added file: http://bugs.python.org/file42192/stat_math.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Jeff Allen]

Changes by Jeff Allen :


Removed file: http://bugs.python.org/file41526/stat_math.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Mark Dickinson]

Mark Dickinson added the comment:

Big +1 for the idea, and thank you for doing this work. I can't promise to find 
time to do a thorough review in the near future (though I'll try).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Jeff Allen]

Jeff Allen added the comment:

Thanks for the prompt acknowledgement and for accepting this to review.

I have updated the coverage & tolerance demo program. Usage in the comments (in 
v3).

I have also added the program I used to generate the extra test cases (needs 
mpmath -- easier to get working than mpf in the original Windows/Jython 
environment).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Jeff Allen]

Changes by Jeff Allen :


Added file: http://bugs.python.org/file42190/stat_math.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +christian.heimes, eric.smith, lemburg, serhiy.storchaka, stutzbach
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26040] Improve coverage and rigour of test.test_math

[Jeff Allen]

Jeff Allen added the comment:

Here is a patch that improves coverage and addresses the uneven accuracy. 
Required accuracy is now specified in ulps. Mostly, I have choses 1 ulp, since 
this passed for me on an x86 architecture (and also ARM), but this may be too 
ambitious.

I have also responded to the comment relating to erfc:
# XXX Would be better to weaken this test only
# for large x, instead of for all x."

I found I could not contribute the code I used to generate the additional test 
cases in Tools/scripts without failing test_tools. (It complained of a missing 
dependency. The generator uses mpmath.)

--
keywords: +patch
Added file: http://bugs.python.org/file42166/iss26040.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com