> >1/meter/39.37 inches = 0.025400051 meters/inch ... > The interesting thing is that this result is off by 2.000004000 ppm. > That's a rather precise error! > > The calculator I used is here: > http://homepage.ntlworld.com/r.howitt/Calculator.htm
David, There are two ways to compute relative error, depending on which one you call the reference. We start with 39.37 * 25.4 = 999.998 (exactly) and note that it is 0.002 (exactly) away from 1000. You can compute the relative error two ways: 1) 0.002 / 1000.0 = 2 ppm (exactly), or 2) 0.002 / 999.998 = your 2.000004000 ppm number. I'd guess convention is to use the first number, not the second. For extra credit, note your number is actually this cool-looking fraction* whose pattern you will immediately recognize. 2.000004000008000016000032000064000128000256000512001024... /tvb http://www.LeapSecond.com * calculated with: echo "80k .002 999.998 / 10 6 ^ * p" | dc _______________________________________________ time-nuts mailing list [email protected] https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
