On Fri, Jul 18, 2014 at 2:29 PM, Nathaniel Smith <[email protected]> wrote:
> On Fri, Jul 18, 2014 at 7:03 PM, <[email protected]> wrote: > > > > On Fri, Jul 18, 2014 at 12:53 PM, Nathaniel Smith <[email protected]> wrote: > >> > >> For cases like this, you need *some* non-zero atol or the thing just > >> doesn't work, and one could quibble over the exact value as long as > >> it's larger than "normal" floating point error. These calculations > >> usually involve "normal" sized numbers, so atol should be comparable > >> to eps * these values. eps is 2e-16, so atol=1e-8 works for values up > >> to around 1e8, which is a plausible upper bound for where people might > >> expect assert_allclose to just work. I'm trying to figure out some way > >> to support your use cases while also supporting other use cases. > > > > > > my problem is that there is no "normal" floating point error. > > If I have units in 1000 or units in 0.0001 depends on the example and > > dataset that we use for testing. > > > > this test two different functions/methods that calculate the same thing > > > > (Pdb) pval > > array([ 3.01270184e-42, 5.90847367e-02, 3.00066946e-12]) > > (Pdb) res2.pvalues > > array([ 3.01270184e-42, 5.90847367e-02, 3.00066946e-12]) > > (Pdb) assert_allclose(pval, res2.pvalues, rtol=5 * rtol, atol=1e-25) > > > > I don't care about errors that are smaller that 1e-25 > > > > for example testing p-values against Stata > > > > (Pdb) tt.pvalue > > array([ 5.70315140e-30, 6.24662551e-02, 5.86024090e-11]) > > (Pdb) res2.pvalues > > array([ 5.70315140e-30, 6.24662551e-02, 5.86024090e-11]) > > (Pdb) tt.pvalue - res2.pvalues > > array([ 2.16612016e-40, 2.51187959e-15, 4.30027936e-21]) > > (Pdb) tt.pvalue / res2.pvalues - 1 > > array([ 3.79811738e-11, 4.01900735e-14, 7.33806349e-11]) > > (Pdb) rtol > > 1e-10 > > (Pdb) assert_allclose(tt.pvalue, res2.pvalues, rtol=5 * rtol) > > > > > > I could find a lot more and maybe nicer examples, since I spend quite a > bit > > of time fine tuning unit tests. > > ...these are all cases where there are not exact zeros, so my proposal > would not affect them? > > I can see the argument that we shouldn't provide any default rtol/atol > at all because there is no good default, but... I don't think putting > that big of a barrier in front of newbies writing their first tests is > a good idea. > I think atol=0 is **very** good for newbies, and everyone else. If expected is really zero or very small, then it immediately causes a test failure, and it's relatively obvious how to fix it. I worry a lot more about unit tests that don't "bite" written by newbies or not so newbies who just use a default. That's one of the problems we had with assert_almost_equal, and why I was very happy to switch to assert_allclose with it's emphasis on relative tolerance. Josef > > -n > > -- > Nathaniel J. Smith > Postdoctoral researcher - Informatics - University of Edinburgh > http://vorpus.org > _______________________________________________ > NumPy-Discussion mailing list > [email protected] > http://mail.scipy.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list [email protected] http://mail.scipy.org/mailman/listinfo/numpy-discussion
