My issue is with relatively small floating point values in the context of OpenGL. More specifically, I ported a matrix inverse calculation and it was producing significantly different results than the C counterpart[1] (which was correct and matched the rendering). I found some of the intermediate values (the determinate) were smaller than 1e6 (my scl is 6) and so my thought is to increase the scale to increase the precision. I will post a brief example in a bit.
[1] https://github.com/felselva/mathc/blob/d672725203fc80f6f79fba64533b87d51c32d714/mathc.c#L3186 On Thu, Dec 9, 2021 at 10:01 AM Alexander Burger <a...@software-lab.de> wrote: > On Thu, Dec 09, 2021 at 09:21:44AM -0500, Kevin Ednalino wrote: > > but I will review the IEEE standard. > > It is not directly an IEEE issue. > > > On Thu, Dec 09, 2021 at 03:36:48PM +0100, Alexander Burger wrote: > > Yes, the number needs to be divided and rounded (with '*/') by the proper > > quotient of both scales (i.e. 1e15 / 1e7). > > > > PicoLisp does not do that, to avoid a permanent runtime overhead. > > If so, it should be done in @src/lib.c, by changing these functions: > > int64_t boxFloat(int32_t value, int64_t scl) { > int64_t n = lroundf(*(float*)&value * (float)scl); > > return n >= 0? n << 4 | 2 : -n << 4 | 10; > } > > int64_t boxDouble(int64_t value, int64_t scl) { > int64_t n = lround(*(double*)&value * (double)scl); > > return n >= 0? n << 4 | 2 : -n << 4 | 10; > } > > void bufFloat(int64_t value, int64_t scl, char* p) { > *(float*)p = (float)number(value) / (float)scl; > } > > void bufDouble(int64_t value, int64_t scl, char* p) { > *(double*)p = (double)number(value) / (double)scl; > } > > The numbers are passed and returned as 64 bit numbers, so we can get an > overflow. > > Is it worth the effort to pass numbers with a higher precision than the > floats > can support? > > ☺/ A!ex > > -- > UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe >
