It depends upon your lat and lon, but figure about 10 feet. Basically earth
circumference is 24,000 miles * 5280 feet per mile divided by (2**24) (bits in
the sign+significand of a 32 bit float). The math works out to 7.5 feet, but
one always has to pay some imprecise math tax...
One can see the quantization effect with Heather V5. Heather can calculate
and display a lat/lon scattergram of the receiver fixes as they come in. The
default is a 10x10 grid at 3 meters per division. The lat/lon coordinates are
"plotted" in an internal bit map. The coordinates are also stored as 32-bit
floats in the plot queue. If one tells Heather to change the scale factor
(meters/division) of the scattergram, it is re-created from the low-res plot
queue data.
gpsd1.gif is from the full-res double precision data the receiver sends,
gpsd2.gif is from the single precision floating point plot queue data. The
center points of two scattergrams are slightly different because it was changed
to the current GPS coordinate when the plot was re-generated. The pixel colors
change every hour.
--------------------
> What does the chop-off translate to in terms of distance?
_______________________________________________
time-nuts mailing list -- [email protected]
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.
