Le 2011-12-08 à 14:59:00, katja a écrit :
Did you try that, Eduardo? In Pd, you get:
100 * (7.1 - 7) / 100 = 0.099
[expr 100*(7.1-7)/100]
|
[makefilename %.21f]
|
[print]
0.09904632568359375
That's actually 209715/2097152. But there aren't so many digits of
precision because once
Thanks Roman, Mathieu et al for the illuminating replies.
On Dec 8, 2011, at 7:57 AM, Mathieu Bouchard ma...@artengine.ca wrote:
Le 2011-12-08 à 00:13:00, Joe Newlin a écrit :
Can someone explain what's going on in the attached example? I'm getting 7.1
minus 7 equals .099. I need to split
On Thu, Dec 8, 2011 at 1:06 PM, Eduardo Flores Abad
m...@eduardoflores.de wrote:
Hi Katja,
tries to round this way:
100 * (7.1 - 7) / 100 = 0.1
Did you try that, Eduardo? In Pd, you get:
100 * (7.1 - 7) / 100 = 0.099
But:
((100 * 7.1) - (100 * 7)) / 100 = 0.1
Or:
((10 * 7.1) -
Le 2011-12-08 à 00:13:00, Joe Newlin a écrit :
Can someone explain what's going on in the attached example? I'm getting
7.1 minus 7 equals .099. I need to split two-digit integers, so this
result is giving me problems.
in fractional powers of two,
1/10 = 1/16 + 1/32 + 1/256 + 1/512 + 1/4096
Can someone explain what's going on in the attached example? I'm getting
7.1 minus 7 equals .099. I need to split two-digit integers, so this result
is giving me problems.
Thanks,
JN
--
www.joenewlin.net
www.twitter.com/joe_newlin
miscalculator.pd
Description: Binary data
Hi Joe
Pd uses 32 bit floating point number format which has (as any other
number format) some limitations. Not every number can be exactly
expressed by this format. '7.1' seems to be such a number.
In case you work with only one digit after the decimal point, I suggest
to multiply your number
On Thu, Dec 8, 2011 at 9:13 AM, Joe Newlin jtnew...@gmail.com wrote:
Can someone explain what's going on in the attached example? I'm getting 7.1
minus 7 equals .099. I need to split two-digit integers, so this result is
giving me problems.
An interesting rounding example. If you resize
