Try using extended precision 2 4 6 %. x:1 2 3 ^/ i.2 0 2
2 4 6 %. 1 2 3 ^/ i.2 > On 01/13/2021 6:56 PM Vanessa McHale <[email protected]> wrote: > > > Hi all, > > I perform the calculation > > 2 4 6 %. 1 2 3 ^/ i.2 > > for the test suite of my j package > https://github.com/vmchale/j-hs/blob/main/test/Spec.hs#L76 > > With J 9.01 this gave (5.995204332975845e-15, 1.9999999999999971) as a > result. With J902, it gives (7.105427357601002e-15, 1.9999999999999967). > > Since the exact answer is (0, 2), this is ever-so-slightly more > incorrect! Any insights into why? > > (I'm on x86_64 and Linux FWIW) > > Cheers, > Vanessa McHale > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
