2015-11-19 3:58 GMT+01:00 Marko Tiikkaja <ma...@joh.to>:
> Here's a patch for the second function suggested in
> 5643125e.1030...@joh.to. This is my first patch trying to do anything
> with numerics, so please be gentle. I'm sure it's full of stupid.
> January's commit fest, feedback welcome, yada yada..
I am looking on this patch and I don't understand to formula
dscale = (ndigits - arg.weight - 1) * DEC_DIGITS;
the following rule is valid
DEC_DIGITS * ndigits >= dscale + arg.weight + 1
so dscale should be calculated like
dscale <= DEC_DIGITS * ndigits - arg.weight - 1
but your formula is correct and working. Can you explain it?