On Thu, Jul 17, 2014 at 4:07 PM, <josef.p...@gmail.com> wrote: > > > > On Wed, Jul 16, 2014 at 9:52 AM, Nathaniel Smith <n...@pobox.com> wrote: > >> On 16 Jul 2014 10:26, "Tony Yu" <tsy...@gmail.com> wrote: >> > >> > Is there any reason why the defaults for `allclose` and >> `assert_allclose` differ? This makes debugging a broken test much more >> difficult. More importantly, using an absolute tolerance of 0 causes >> failures for some common cases. For example, if two values are very close >> to zero, a test will fail: >> > >> > np.testing.assert_allclose(0, 1e-14) >> > >> > Git blame suggests the change was made in the following commit, but I >> guess that change only reverted to the original behavior. >> > >> > >> https://github.com/numpy/numpy/commit/f43223479f917e404e724e6a3df27aa701e6d6bf >> > >> > It seems like the defaults for `allclose` and `assert_allclose` should >> match, and an absolute tolerance of 0 is probably not ideal. I guess this >> is a pretty big behavioral change, but the current default for >> `assert_allclose` doesn't seem ideal. >> >> What you say makes sense to me, and loosening the default tolerances >> won't break any existing tests. (And I'm not too worried about people who >> were counting on getting 1e-7 instead of 1e-5 or whatever... if it matters >> that much to you exactly what tolerance you test, you should be setting the >> tolerance explicitly!) I vote that unless someone comes up with some >> terrible objection in the next few days then you should submit a PR :-) >> > > If you mean by this to add atol=1e-8 as default, then I'm against it. > > At least it will change the meaning of many of our tests in statsmodels. > > I'm using rtol to check for correct 1e-15 or 1e-30, which would be > completely swamped if you change the default atol=0. > Adding atol=0 to all assert_allclose that currently use only rtol is a lot > of work. > I think I almost never use a default rtol, but I often leave atol at the > default = 0. > > If we have zeros, then I don't think it's too much work to decide whether > this should be atol=1e-20, or 1e-8. >
Just to explain, p-values, sf of the distributions are usually accurate at 1e-30 or 1e-50 or something like that. And when we test the tails of the distributions we use that the relative error is small and the absolute error is "tiny". We would need to do a grep to see how many cases there actually are in scipy and statsmodels, before we change it because for some use cases we only get atol 1e-5 or 1e-7 (e.g. nonlinear optimization). Linear algebra is usually atol or rtol 1e-11 to 1e-14 in my cases, AFAIR. Josef > > Josef > > > >> -n >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> >> >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
