-------- In message <cajgz9aygqqjomok57ybiohf6qh0dx6seh-m_rxabwqc54ud...@mail.gmail.com>, Russ Ramirez writes:
>My question is this. If I measured a 10 volt reference known to be good to >8 1/2 digits, but with the LSD being > 0, assuming a transfer standard with >traceability and documented uncertainties etc, would a 6 1/2 digit >voltmeter read 10.00000 volts, or round up to the limits of its AD >converter resolution; say X uV over 10.00000 volts? Somewhere in your instrument there is an A/D converter of some kind. Even if it is perfect, you can never know if the last bit from the A/D represents a voltage which is slightly higher or slightly lower than the number it reads. This is why specifications for digital instruments always includes "± 1 digit" In theory, if you have *truly* perfect electronics, it could be "± 0.5 digit", because in that case you might as well use the same ±1 as everybody else and double the size of the range. The A/D in a 6½ digit instrument by definition must have at least 22 bits (2^21 = 2097152 + sign bit) and if we scale things so the least significant bit is 5 microvolts, we have a full scale voltage of 2^21 * 5e-6V = 10.48576V. QED: Uncertainty on your 6½ digit meter on a 10V signal will always be at least ± 5 microvolts. If you do the same math for a true 8½ digit DVM, you need a 29 bit A/D converter (2^28 = 268435456 + sign) which means you can use 40 nV as stepsize yielding 10.73V range, and thus in teory get ± 40nV measurements. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [email protected] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. _______________________________________________ volt-nuts mailing list -- [email protected] To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/volt-nuts and follow the instructions there.
