Re: [polyml] PackReal32

2021-02-21 Thread Chris Cannam
I took another little look at the behaviour here. My basic assumption is that, for any Real32.real r, Real32.== (PackReal32Big.fromBytes (PackReal32Big.toBytes r), r) should hold. (and the same for Little obviously) With the current repo code, it's possible to construct values for which this

Re: [polyml] PackReal32

2021-02-21 Thread David Matthews
Thank you for looking at this. You're quite right that the shifts were wrong. I've pushed a fix for this that seems to have corrected it. I'm not that familiar with the intricacies of floating point numbers so I'm always glad when someone points out problems. David On 21/02/2021 13:50,