On Mon, 17 Dec 2007, marius schebella wrote:
for now, I am ok with a precision of 1cm.
You won't get it with just floats. You will have to use pairs of floats to
get to this level.
but what is highest positive integer number I can represent with the current
pd float precision? 16bit?
Mathieu Bouchard wrote:
but what is highest positive integer number I can represent with the
current pd float precision? 16bit? 32.767?
the highest positive integer is 340282346638528859811704183484516925440
but you really want to know which is the highest positive integer for
which it
On Thu, 13 Dec 2007, Mathieu Bouchard wrote:
You can get twice better worst case by using values ranging from -1 to +1
(because 1 is a power of two, so it lies at the boundary of a new precision
level)
There was also a mistake in saying that. It improves precision, but only
by a factor of
Miller Puckette wrote:
If I'm doing it right, single precision float should be able to represent
latitude and longitude to within about two meters.
yes, a precision of 2m is just not enough for showing buildings or streets.
If more precision than that is needed, you'll want to use tr to
for now, I am ok with a precision of 1cm.
but what is highest positive integer number I can represent with the
current pd float precision? 16bit? 32.767? (btw., would be nice to have
this information in the float/number helppatch)
that would only give me a precision of around 3.4 meters...
hi,
google earth uses a special format to save geo information data (kml
files). I am trying to build a (simple) gem earth projector and read
these files (and also some other file types...)
The conversion should be easy, but precision might become a problem.
this is a shape in kml file format
If I'm doing it right, single precision float should be able to represent
latitude and longitude to within about two meters.
If more precision than that is needed, you'll want to use tr to change
periods (as well as commas) into spaces so that you get lines like:
-112 3348783983763 36
On Thu, 13 Dec 2007, Miller Puckette wrote:
If I'm doing it right, single precision float should be able to represent
latitude and longitude to within about two meters.
longitude has to be from -180 to 180. The epsilon is then the previous
power of two divided by 2^23. In metres this is 0.61
On Thu, 13 Dec 2007, Mathieu Bouchard wrote:
longitude has to be from -180 to 180. The epsilon is then the previous power
of two divided by 2^23. In metres this is 0.61 metre near equator. This is
the worst case. For latitude the precision is twice better than longitude at
equator. In
