Re: [Haskell-cafe] attoparsec double precision, quickCheck and aeson
Bryan, do you remember what the issue is with C++ in this case? I thought, adding a wrapper with extern C definitions should do the trick for simpler libraries (as this one seems to be). Is the interaction with the memory allocator the issue? Linker flags? On 11 June 2012 06:38, Bryan O'Sullivan b...@serpentine.com wrote: On Wed, Jun 6, 2012 at 6:20 AM, Doug McIlroy d...@cs.dartmouth.edu wrote: Last I looked (admittedly quite a while ago), the state of the art was strtod in http://www.netlib.org/fp/dtoa.c. (Alas, dtoa.c achieves calculational perfection via a murmuration of #ifdefs.) That was indeed the state of the art for about three decades, until Florian Loitsch showed up in 2010 with an algorithm that is usually far faster: http://www.serpentine.com/blog/2011/06/29/here-be-dragons-advances-in-problems-you-didnt-even-know-you-had/ Unfortunately, although I've written Haskell bindings to his library, said library is written in C++, and our FFI support for C++ libraries is negligible and buggy. As a result, that code is disabled by default. It's disheartening to hear that important Haskell code has needlessly fallen from perfection--perhaps even deliberately. Indeed (and yes, it's deliberate). If I had the time to spare, I'd attempt to fix the situation by porting Loitsch's algorithm to Haskell or C, but either one would be a lot of work - the library is 5,600 lines of tricky code. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Push the envelope. Watch it bend. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] attoparsec double precision, quickCheck and aeson
On Mon, Jun 11, 2012 at 10:50 AM, Thomas Schilling nomin...@googlemail.comwrote: Bryan, do you remember what the issue is with C++ in this case? I thought, adding a wrapper with extern C definitions should do the trick for simpler libraries (as this one seems to be). Is the interaction with the memory allocator the issue? Linker flags? It's specific to ghci, whose object file loader fails to call C++ static initializers. In the case of the double-conversion library, this means that static read-only arrays that it assumes to contain valid data are full of junk. You can join in the fun over at http://hackage.haskell.org/trac/ghc/ticket/5289 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] attoparsec double precision, quickCheck and aeson
On Mon, Jun 11, 2012 at 10:57 AM, Bryan O'Sullivan b...@serpentine.comwrote: In the case of the double-conversion library, this means that static read-only arrays that it assumes to contain valid data are full of junk. You can join in the fun over at http://hackage.haskell.org/trac/ghc/ticket/5289 Oops, that bug is not actually relevant to this case. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] attoparsec double precision, quickCheck and aeson
On Wed, Jun 6, 2012 at 6:20 AM, Doug McIlroy d...@cs.dartmouth.eduwrote: Last I looked (admittedly quite a while ago), the state of the art was strtod in http://www.netlib.org/fp/dtoa.c. (Alas, dtoa.c achieves calculational perfection via a murmuration of #ifdefs.) That was indeed the state of the art for about three decades, until Florian Loitsch showed up in 2010 with an algorithm that is usually far faster: http://www.serpentine.com/blog/2011/06/29/here-be-dragons-advances-in-problems-you-didnt-even-know-you-had/ Unfortunately, although I've written Haskell bindings to his library, said library is written in C++, and our FFI support for C++ libraries is negligible and buggy. As a result, that code is disabled by default. It's disheartening to hear that important Haskell code has needlessly fallen from perfection--perhaps even deliberately. Indeed (and yes, it's deliberate). If I had the time to spare, I'd attempt to fix the situation by porting Loitsch's algorithm to Haskell or C, but either one would be a lot of work - the library is 5,600 lines of tricky code. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] attoparsec double precision, quickCheck and aeson
Date: Tue, 5 Jun 2012 10:25:26 -0700 From: Warren Harris warrensomeb...@gmail.com On Jun 5, 2012, at 9:57 AM, Johan Tibell wrote: I don't think applying == to something that contains floating point values at the leaves makes much sense. You want some approxEq function that uses approximate equality on floating point value or you want to equality function that ignores the floating point values. Probably not the answer you like, but I don't know how to define Eq in a robust way for types that include floating point values. I buy that in general for comparing floats (those that result from arithmetic operations), but this is a case where attoparsec's parser is munging the value. I would like to have a law that says parse . print == id ... which is why this seems more like a bug than the usual floating point concerns. This law seems to hold for haskell's double parser: quickCheck (\d - read (show d) == d) Date: Tue, 5 Jun 2012 10:51:08 -0700 From: Bryan O'Sullivan b...@serpentine.com If you need the full precision, use rational instead. The double parser is there because parsing floating point numbers is often a bottleneck, and double intentionally trades speed for precision. If I understand the intended meaning of parse correctly, what's at issue is decimal-to-binary conversion. It is hard to defend sloppy answers for such a fundamental operation. The recommendation of rational calculation makes little sense for most floating-point computation, which approximates irrationals. The necessary imprecision, though, does not justify sloppiness--and especially not sloppy tests. It's worth noting that a law of input being inverse to output must fail, though rarely. Different granularity of the two bases means that there must exist cases where adjacent values in one base convert to the same value in the other. (Conceivably the specialization to parse.print could hold for some hardware. Does anybody know whether the wish is actually hopeless?) Last I looked (admittedly quite a while ago), the state of the art was strtod in http://www.netlib.org/fp/dtoa.c. (Alas, dtoa.c achieves calculational perfection via a murmuration of #ifdefs.) It's disheartening to hear that important Haskell code has needlessly fallen from perfection--perhaps even deliberately. Doug McIlroy ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] attoparsec double precision, quickCheck and aeson
The double parser provided by Data.Attoparsec.ByteString.Char8 looses precision around the 13-15th decimal place (http://hackage.haskell.org/packages/archive/attoparsec/0.10.2.0/doc/html/Data-Attoparsec-ByteString-Char8.html#v:double). Unfortunately this reeks havoc with my attempts to write a quickCheck test that validates print-read equivalence for a program that uses Aeson. I have tried compensating for this round-off error in my quickCheck generator using a function like this: roundDouble :: Double - Double roundDouble d = let Right v = A8.parseOnly A8.double (C.pack (show d)) in v which helps in many cases, but for some the parsing seems bi-stable, alternating between two imprecise double values and causing the test to fail. I was wondering if anyone could suggest a better work-around for this problem, or explain why Attoparsec's double parser can't be isomorphic to haskell's. Thanks, Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] attoparsec double precision, quickCheck and aeson
On Tue, Jun 5, 2012 at 9:12 AM, Warren Harris warrensomeb...@gmail.com wrote: which helps in many cases, but for some the parsing seems bi-stable, alternating between two imprecise double values and causing the test to fail. You want to perform your test as d1 - d2 epsilon where epsilon is derived from the relative error you're willing to accept (e.g. 0.01% error.) You can't use an absolute epsilon because if you pick e.g. epsilon = 0.1 but your input are also very small, you'll end up accepting a big relative error. Comparing floating points values for equality is asking for trouble. I was wondering if anyone could suggest a better work-around for this problem, or explain why Attoparsec's double parser can't be isomorphic to haskell's. Thanks, I think attoparsec uses a slightly less exact but much much faster double parser. I believe it's described in the attoparsec haddock docs somewhere. -- Johan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] attoparsec double precision, quickCheck and aeson
On Jun 5, 2012, at 9:38 AM, Johan Tibell wrote: You want to perform your test as d1 - d2 epsilon What's the best way to do this though, since aeson's Value type already provides instance Eq? I guess I can write my own suite of equality comparisons, but these aeson values are the leaves of an extensive grammar and it seems a shame to mimic everything that Eq does (without the benefits of deriving) just to compare doubles differently. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] attoparsec double precision, quickCheck and aeson
On Tue, Jun 5, 2012 at 9:51 AM, Warren Harris warrensomeb...@gmail.com wrote: On Jun 5, 2012, at 9:38 AM, Johan Tibell wrote: You want to perform your test as d1 - d2 epsilon What's the best way to do this though, since aeson's Value type already provides instance Eq? I guess I can write my own suite of equality comparisons, but these aeson values are the leaves of an extensive grammar and it seems a shame to mimic everything that Eq does (without the benefits of deriving) just to compare doubles differently. I don't think applying == to something that contains floating point values at the leaves makes much sense. You want some approxEq function that uses approximate equality on floating point value or you want to equality function that ignores the floating point values. Probably not the answer you like, but I don't know how to define Eq in a robust way for types that include floating point values. -- Johan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] attoparsec double precision, quickCheck and aeson
On Jun 5, 2012, at 9:57 AM, Johan Tibell wrote: I don't think applying == to something that contains floating point values at the leaves makes much sense. You want some approxEq function that uses approximate equality on floating point value or you want to equality function that ignores the floating point values. Probably not the answer you like, but I don't know how to define Eq in a robust way for types that include floating point values. I buy that in general for comparing floats (those that result from arithmetic operations), but this is a case where attoparsec's parser is munging the value. I would like to have a law that says parse . print == id ... which is why this seems more like a bug than the usual floating point concerns. This law seems to hold for haskell's double parser: quickCheck (\d - read (show d) == d) ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] attoparsec double precision, quickCheck and aeson
On Tue, Jun 5, 2012 at 9:12 AM, Warren Harris warrensomeb...@gmail.comwrote: which helps in many cases, but for some the parsing seems bi-stable, alternating between two imprecise double values and causing the test to fail. I was wondering if anyone could suggest a better work-around for this problem, or explain why Attoparsec's double parser can't be isomorphic to haskell's. If you need the full precision, use rational instead. The double parser is there because parsing floating point numbers is often a bottleneck, and double intentionally trades speed for precision. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] attoparsec double precision, quickCheck and aeson
On Tue, Jun 5, 2012 at 10:51 AM, Bryan O'Sullivan b...@serpentine.com wrote: If you need the full precision, use rational instead. The double parser is there because parsing floating point numbers is often a bottleneck, and double intentionally trades speed for precision. Relevant code in https://github.com/bos/attoparsec/blob/master/Data/Attoparsec/ByteString/Char8.hs -- Johan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] attoparsec double precision, quickCheck and aeson
On Jun 5, 2012, at 10:51 AM, Bryan O'Sullivan wrote: On Tue, Jun 5, 2012 at 9:12 AM, Warren Harris warrensomeb...@gmail.com wrote: which helps in many cases, but for some the parsing seems bi-stable, alternating between two imprecise double values and causing the test to fail. I was wondering if anyone could suggest a better work-around for this problem, or explain why Attoparsec's double parser can't be isomorphic to haskell's. If you need the full precision, use rational instead. The double parser is there because parsing floating point numbers is often a bottleneck, and double intentionally trades speed for precision. I'm actually using Aeson and the parser it provides, so I don't really have the option of using rational.___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
