#10952: better numerical accuracy testing
-----------------------------------------------------------------------+----
Reporter: robertwb |
Owner: mvngu
Type: enhancement |
Status: needs_review
Priority: critical |
Milestone: sage-4.7.2
Component: doctest |
Keywords: sd32
Work_issues: |
Upstream: N/A
Reviewer: Jason Grout, Mariah Lenox, William Stein, John Palmieri |
Author: Robert Bradshaw, Rob Beezer
Merged: |
Dependencies:
-----------------------------------------------------------------------+----
Old description:
> If a line contains `tol` or `tolerance`, numerical results are only
> verified to the given tolerance. This may be prefixed by `abs[olute]` or
> `rel[ative]` to specify whether to measure absolute or relative error;
> defaults to relative error except when the expected value is exactly
> zero:
>
> {{{
> sage: RDF(pi) # abs tol 1e-5
> 3.14159
> sage: [10^n for n in [0.0 .. 4]] # rel tol 2e-4
> [0.9999, 10.001, 100.01, 999.9, 10001]
> }}}
>
> This can be useful when the exact output is subject to rounding error
> and/or processor floating point arithmetic variation.
>
> ----
>
> Apply
> 1. [attachment:10952-tol-bin.2.patch]
> to the Sage '''scripts repository'''.
>
> Apply
> 1. [attachment:10952-tol-doc.2.patch]
> 1. [attachment:trac_10952-reviewer-docs-v3.patch]
> to the Sage library repository.
New description:
If a line contains `tol` or `tolerance`, numerical results are only
verified to the given tolerance. This may be prefixed by `abs[olute]` or
`rel[ative]` to specify whether to measure absolute or relative error;
defaults to relative error except when the expected value is exactly zero:
{{{
sage: RDF(pi) # abs tol 1e-5
3.14159
sage: [10^n for n in [0.0 .. 4]] # rel tol 2e-4
[0.9999, 10.001, 100.01, 999.9, 10001]
}}}
This can be useful when the exact output is subject to rounding error
and/or processor floating point arithmetic variation.
----
Apply
1. [attachment:10952-tol-bin.2.patch]
1. [attachment:trac_10952-ref.patch]
to the Sage '''scripts repository'''.
Apply
1. [attachment:10952-tol-doc.2.patch]
1. [attachment:trac_10952-reviewer-docs-v3.patch]
to the Sage library repository.
--
Comment(by jhpalmieri):
I don't see anything wrong with more complicated regular expressions, but
I like regular expressions. So leave it as is if you want.
Meanwhile, the output for failures using tolerances isn't nice, compared
to other sorts of failures:
{{{
sage -t "builds/sage-4.7.2.alpha2/devel/sage-new/sage/homology/new.py"
**********************************************************************
File "/Applications/sage_builds/sage-4.7.2.alpha2/devel/sage-
new/sage/homology/new.py", line 4:
sage: 3+3
Expected:
8
Got:
6
**********************************************************************
File "/Applications/sage_builds/sage-4.7.2.alpha2/devel/sage-
new/sage/homology/new.py", iled example:
check_with_tolerance('''
3.14159
''', res, 9.9999999999999998e-13, 'abs')
Exception raised:
Traceback (most recent call last):
File "/Applications/sage/local/bin/ncadoctest.py", line 1231, in
run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/Applications/sage/local/bin/sagedoctest.py", line 38, in
run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename,
compileflags)
File "/Applications/sage/local/bin/ncadoctest.py", line 1172, in
run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_0[11]>", line 3, in <module>
''', res, 9.9999999999999998e-13, 'abs')
File "/Users/palmieri/.sage//tmp/new.py", line 35, in
check_with_tolerance
assert abs(expected_value - actual_value) < epsilon, "Out of
tolerance %s vs %s" % (expected_value, actual_value)
AssertionError: Out of tolerance 3.14159 vs 3.14159265359
**********************************************************************
1 items had failures:
2 of 13 in __main__.example_0
***Test Failed*** 2 failures.
For whitespace errors, see the file
/Users/palmieri/.sage//tmp/.doctest_new.py
[3.9 s]
}}}
Notice that the line number is missing and the traceback is present. I'm
attaching a referee patch which tries to fix this. Please take a look.
For me, this gives this sort of output (with a slightly different file):
{{{
sage -t "builds/sage-4.7.2.alpha2/devel/sage-new/new.py"
**********************************************************************
File "/Applications/sage_builds/sage-4.7.2.alpha2/devel/sage-new/new.py",
line 4:
sage: 3+3
Expected:
8
Got:
6
**********************************************************************
File "/Applications/sage_builds/sage-4.7.2.alpha2/devel/sage-new/new.py",
line 6:
sage: RDF(pi) # abs tol 1e-6
Out of tolerance 3.14159 vs 3.14159265359
**********************************************************************
File "/Applications/sage_builds/sage-4.7.2.alpha2/devel/sage-new/new.py",
line 16:
sage: RDF(pi) # abs tol 1e-8
Out of tolerance 3.14159 vs 3.14159265359
**********************************************************************
1 items had failures:
3 of 16 in __main__.example_0
***Test Failed*** 3 failures.
For whitespace errors, see the file
/Users/palmieri/.sage//tmp/.doctest_new.py
[4.2 s]
}}}
Since the whole ticket had a positive review earlier, I think if you're
happy with my changes, you can switch it back to that status.
Finally, the issue with doctests in `conventions.rst` is, I think, that
the various triple quotes `"""` confuse the parsing routine (the function
`pythonify_rst` in particular). This belongs on another ticket. That
file contains enough incomplete code stubs that I wonder if it shouldn't
be doctested at all...
--
Ticket URL: <http://trac.sagemath.org/sage_trac/ticket/10952#comment:39>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sage-trac?hl=en.
