For the applications I work on, double precision floats are too costly to use; although the CPU cycle count to operate on doubles tend to be the same as single precision floats on modern hardware, the bandwidth cost is too prohibitive. We really do need single precision floats, and in many cases, 16 bit half precision floats due to the bandwidth savings.
With regard to exactness, I don't need exactness to compare two single precision floats. I would like to have exactness in the ground truth that I compute to be able to calculate the error in the single precision float version of the computation. The idea is that I implement two versions of an algorithm. One uses the exact numbers supported by Racket and the other would use single precision floats, then I would like to compute error with (flulp-error x r) or something similar. Is there a better approach to do this kind of analysis? -Dale Kim On Monday, April 9, 2018 at 10:13:17 PM UTC-7, gneuner2 wrote: > > > On 4/9/2018 7:11 PM, dk...@insomniacgames.com <javascript:> wrote: > > I'm very interested in using Racket for the purposes of numerical > analysis. Specifically, I am interested in using Racket as my test bed for > implementing simple numerical algorithms which operate on IEEE 754 single > precision floats and compare those results against a ground truth, ideally, > the exact result. Racket appears to have great support for double precision > floats, but I haven't found any functions that correspond to single > precision floats. > > Some questions: > > 1. Should I be implementing my own versions of the flonum arithmetic > functions that operate on single precision floats? (See these functions: > > https://docs.racket-lang.org/reference/flonums.html#%28part._.Flonum_.Arithmetic%29 > ) > 2. What about other support functions that require knowledge of the > bit patterns of the floats? (See > > https://docs.racket-lang.org/math/flonum.html#%28part._.Measuring_.Floating-.Point_.Error%29 > > or > > https://docs.racket-lang.org/math/flonum.html#%28part._.Low-.Level_.Flonum_.Operations%29 > ) > > > As Philip McGrath mentioned already, you can specify single precision at > least for input values ... but I think inexact (i.e. floating point) math > in Racket all is done at maximum hardware precision and the result coerced > back to the input precision if possible. For purposes of numerical > analysis, that is not the same as performing all operations at the input > precision. > > Not kidding, if you REALLY need single precision your best bet might be to > build a custom version of Racket. But on most hardware today, single > precision is not any faster than double, so if your concern is about speed > it is mostly unfounded. If, OTOH, your concern is being able to follow the > bits using pencil and paper ... then I understand the desire for smaller > representation. 8-) > > > But your comment about exactness is a bit troubling. You need to realize > that, in general, floating point values can't be compared for equality. > Floating point - regardless of precision or base - is problematic because > any limited width representation will be unable to exactly represent a > significant fraction of the numbers within the range. It doesn't matter > how many bits are available. Not to mention that IEEE 754 defines 2 zeros, > 2 infinities, and multiple unrepresentable values (NaNs) that you may or > may not need to distinguish. > > See: David Goldberg, "What every computer scientist should know about > floating-point arithmetic" > HTML: https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html > PDF: http://perso.ens-lyon.fr/jean-michel.muller/goldberg.pdf > > Numerical analysis, in general, is not about getting exact answers - > rather it is about minimizing error when exact answers either are not > possible, or are not practical to compute. > > > I would *love* to use Racket for this task and ideally, I need to have > access to all the functions on the pages linked above but for single > precision floats > > (half precision floats would also be nice!). > > > Racket has no support for half precision numbers. Jay McCarthy has made > a binding library available for an old(er) version of OpenCL, but I don't > know its capabilities. > > > Have I missed something obvious? > > -Dale Kim > > > Hope this helps, > George > > On Monday, April 9, 2018 at 10:13:17 PM UTC-7, gneuner2 wrote: > > > On 4/9/2018 7:11 PM, dk...@insomniacgames.com <javascript:> wrote: > > I'm very interested in using Racket for the purposes of numerical > analysis. Specifically, I am interested in using Racket as my test bed for > implementing simple numerical algorithms which operate on IEEE 754 single > precision floats and compare those results against a ground truth, ideally, > the exact result. Racket appears to have great support for double precision > floats, but I haven't found any functions that correspond to single > precision floats. > > Some questions: > > 1. Should I be implementing my own versions of the flonum arithmetic > functions that operate on single precision floats? (See these functions: > > https://docs.racket-lang.org/reference/flonums.html#%28part._.Flonum_.Arithmetic%29 > ) > 2. What about other support functions that require knowledge of the > bit patterns of the floats? (See > > https://docs.racket-lang.org/math/flonum.html#%28part._.Measuring_.Floating-.Point_.Error%29 > > or > > https://docs.racket-lang.org/math/flonum.html#%28part._.Low-.Level_.Flonum_.Operations%29 > ) > > > As Philip McGrath mentioned already, you can specify single precision at > least for input values ... but I think inexact (i.e. floating point) math > in Racket all is done at maximum hardware precision and the result coerced > back to the input precision if possible. For purposes of numerical > analysis, that is not the same as performing all operations at the input > precision. > > Not kidding, if you REALLY need single precision your best bet might be to > build a custom version of Racket. But on most hardware today, single > precision is not any faster than double, so if your concern is about speed > it is mostly unfounded. If, OTOH, your concern is being able to follow the > bits using pencil and paper ... then I understand the desire for smaller > representation. 8-) > > > But your comment about exactness is a bit troubling. You need to realize > that, in general, floating point values can't be compared for equality. > Floating point - regardless of precision or base - is problematic because > any limited width representation will be unable to exactly represent a > significant fraction of the numbers within the range. It doesn't matter > how many bits are available. Not to mention that IEEE 754 defines 2 zeros, > 2 infinities, and multiple unrepresentable values (NaNs) that you may or > may not need to distinguish. > > See: David Goldberg, "What every computer scientist should know about > floating-point arithmetic" > HTML: https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html > PDF: http://perso.ens-lyon.fr/jean-michel.muller/goldberg.pdf > > Numerical analysis, in general, is not about getting exact answers - > rather it is about minimizing error when exact answers either are not > possible, or are not practical to compute. > > > I would *love* to use Racket for this task and ideally, I need to have > access to all the functions on the pages linked above but for single > precision floats > > (half precision floats would also be nice!). > > > Racket has no support for half precision numbers. Jay McCarthy has made > a binding library available for an old(er) version of OpenCL, but I don't > know its capabilities. > > > Have I missed something obvious? > > -Dale Kim > > > Hope this helps, > George > > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
