Re: String to Float, without introducing errors

[Ángel GR]

On 2022-12-19 16:14, MRAB wrote:
To be fair, I don't think I've never seen that notation either! I've 
only ever seen the form 6.67430E-11 ± 0.00015E-11, which is much clearer.


We use it regularly in our experimental data: 6.3(4), 15.002(10). Things 
would become complex using exponential forms for errors, specially when 
starting to play with exponents: 6.67430E-11 ± 1.5E-15.

--
Ángel GR

--
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Michael F. Stemper]

On 19/12/2022 09.14, MRAB wrote:

On 2022-12-19 14:10, Peter J. Holzer wrote:

On 2022-12-19 09:25:17 +1100, Chris Angelico wrote:

On Mon, 19 Dec 2022 at 07:57, Stefan Ram  wrote:
> G = Decimal( 6.6743015E-11 )
> r = Decimal( 6.371E6 )
> M = Decimal( 5.9722E24 )

What's the point of using Decimal if you start with nothing more than
float accuracy?


Right. He also interpreted the notation "6.67430(15)E-11" wrong. The
digits in parentheses represent the uncertainty in the same number of
last digits. So "6.67430(15)E-11" means "something between 6.67430E-11 -
0.00015E-11 and 6.67430E-11 + 0.00015E-11". The r value has only a
precision of 1 km and I'm not sure how accurate the mass is. Let's just
assume (for the sake of the argument) that these are actually accurate in
all given digits.


ntal misunderstanding of the numbers they are working with.



To be fair, I don't think I've never seen that notation either! I've only ever 
seen the form 6.67430E-11 ± 0.00015E-11, which is much clearer.


See, for instance:

In particular, the "concise form".

For more detail, see:


--
Michael F. Stemper
Isaiah 58:6-7
--
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Sabrina Almodóvar]

On 17/12/2022 18:55, Stefan Ram wrote:
> Grant Edwards  writes:
>> Yes, fixed point (or decimal) is a better fit for what he's doing. but
>> I suspect that floating point would be a better fit for the problem
>> he's trying to solve.
> 
>   I'd like to predict that within the next ten posts in this
>   thread someone will mention "What Every Computer Scientist
>   Should Know About Floating-Point Arithmetic".

Lol!  Speaking of which, let us guide ourselves by

  https://floating-point-gui.de/basic/

and learn that calculations such as ``0.1 + 0.4 work correctly.''
That's the Internet.  You learn a thing on a Tuesday, and on a Wednesday
you're a teacher.  That's why we need peer review.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Greg Ewing]

On 19/12/22 9:24 am, Stefan Ram wrote:

 So what's the time until a mass of one gram
 arrives at the ground versus a mass of ten grams? I think
 one needs "Decimal" to calculate this!


Or you can be smarter about how you calculate it.

Differentiating t with respect to m gives

dt/dm = -0.5 * sqrt(2 * s * r**2 / (G * (M + m)**3))

which, since m is much smaller than M, is approximately

   -0.5 * sqrt(2 * s * r**2 / (G * M**3))

So

>>> G = 6.6743015E-11
>>> r = 6.371E6
>>> M = 5.9722E24
>>> dtdm = -0.5 * sqrt(2*s*(r**2) / (G * M**3))
>>> dtdm * (1/1000 - 10/1000)
3.4004053539917275e-28

which agrees with your Decimal calculation to 3 digits,
and should be as precise as the input numbers (about
4 digits in this case).

This is a good example of why it's important to choose
an appropriate numerical algorithm!

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Peter J. Holzer]

On 2022-12-19 15:14:14 +, MRAB wrote:
> On 2022-12-19 14:10, Peter J. Holzer wrote:
> > He also interpreted the notation "6.67430(15)E-11" wrong. The
> > digits in parentheses represent the uncertainty in the same number of
> > last digits. So "6.67430(15)E-11" means "something between 6.67430E-11 -
> > 0.00015E-11 and 6.67430E-11 + 0.00015E-11".
[...]
> To be fair, I don't think I've never seen that notation either!

I've probably seen it first on Wikipedia, quite a few years ago. Since
then I've also encountered in in physical and astronomical papers (I'm
neither a physicist nor an astronomomer but I occasionally read the
original papers if what I read in the "mainstream media"[1] or hear on
youtube seems suspect).

> I've only ever seen the form 6.67430E-11 ± 0.00015E-11, which is much
> clearer.

Yeah, it's definitely not the pinnacle of inuitiveness. I freely admit
that I looked it up before posting just to make sure that I wasn't
confused about its meaning.

Another problem (but it shares that with the ± notation) is that it's
not clear what that number actually represents. Is it one sigma or two?
Or something else? Is the distribution even symmetric?

hp

[1] I'm not quite happy with that term.

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[MRAB]

On 2022-12-19 14:10, Peter J. Holzer wrote:

On 2022-12-19 09:25:17 +1100, Chris Angelico wrote:

On Mon, 19 Dec 2022 at 07:57, Stefan Ram  wrote:
> G = Decimal( 6.6743015E-11 )
> r = Decimal( 6.371E6 )
> M = Decimal( 5.9722E24 )

What's the point of using Decimal if you start with nothing more than
float accuracy?


Right. He also interpreted the notation "6.67430(15)E-11" wrong. The
digits in parentheses represent the uncertainty in the same number of
last digits. So "6.67430(15)E-11" means "something between 6.67430E-11 -
0.00015E-11 and 6.67430E-11 + 0.00015E-11". The r value has only a
precision of 1 km and I'm not sure how accurate the mass is. Let's just
assume (for the sake of the argument) that these are actually accurate in
all given digits.

So G is between 6.67415E-11 and 6.67445E-11, r is between 6.3705E6 and
6.3715E6 and M is between 5.97215E24 and 5.97225E24. If we compute the
time for those deviations you will find that the differences are many
orders of magnitude greater than the effect you wanted to show. And that
still ignores the fact that a vacuum won't be perfect (and collisions
with a few stray atoms might have a similarly tiny effect), that gravity
isn't constant while the weight falls (it's getting closer to the center
of the earth and it's moving past other masses on its way) that Newton's
law is only an approximation, etc. So while the effect is (almost
certainly) real, the numbers are garbage.

I think there's a basic numeracy problem here. This is unfortunately all
too common, even among scientists. The OP apparently rounded their
numbers to 8 significant digits (thereby introducing an error of about
1E-8) and then insisted that the additional error of 1E-15 introduced by
the decimal to float conversion was unacceptable, showing IMHO a
fundamental misunderstanding of the numbers they are working with.

To be fair, I don't think I've never seen that notation either! I've 
only ever seen the form 6.67430E-11 ± 0.00015E-11, which is much clearer.

--
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Thomas Passin]

On 12/19/2022 9:10 AM, Peter J. Holzer wrote:

On 2022-12-19 09:25:17 +1100, Chris Angelico wrote:

On Mon, 19 Dec 2022 at 07:57, Stefan Ram  wrote:

G = Decimal( 6.6743015E-11 )
r = Decimal( 6.371E6 )
M = Decimal( 5.9722E24 )


What's the point of using Decimal if you start with nothing more than
float accuracy?


Right. He also interpreted the notation "6.67430(15)E-11" wrong. The
digits in parentheses represent the uncertainty in the same number of
last digits. So "6.67430(15)E-11" means "something between 6.67430E-11 -
0.00015E-11 and 6.67430E-11 + 0.00015E-11". The r value has only a
precision of 1 km and I'm not sure how accurate the mass is. Let's just
assume (for the sake of the argument) that these are actually accurate in
all given digits.

So G is between 6.67415E-11 and 6.67445E-11, r is between 6.3705E6 and
6.3715E6 and M is between 5.97215E24 and 5.97225E24. If we compute the
time for those deviations you will find that the differences are many
orders of magnitude greater than the effect you wanted to show. And that
still ignores the fact that a vacuum won't be perfect (and collisions
with a few stray atoms might have a similarly tiny effect), that gravity
isn't constant while the weight falls (it's getting closer to the center
of the earth and it's moving past other masses on its way) that Newton's
law is only an approximation, etc. So while the effect is (almost
certainly) real, the numbers are garbage.

I think there's a basic numeracy problem here. This is unfortunately all
too common, even among scientists. The OP apparently rounded their
numbers to 8 significant digits (thereby introducing an error of about
1E-8) and then insisted that the additional error of 1E-15 introduced by
the decimal to float conversion was unacceptable, showing IMHO a
fundamental misunderstanding of the numbers they are working with.

 hp


In a way, this example shows both things - the potential value of using 
Decimal numbers, and a degree of innumeracy.  It also misses a chance to 
illustrate how to approach a problem in the simplest and most 
informative way.  Here's what I mean -


We can imagine that the input numbers really are exact, with the 
remaining digits filled in with zeros.  Then it might really be the case 
that - if you wanted to do this computation with precision and could 
assume all those other effects could be neglected - using Decimals would 
be a good thing to do. So OK, let's say that's demonstrated.  No need to 
nit-pick it further.


As a physics problem, though, you would generally be interested in two 
kinds of things:


1. Could there be such an effect, and if so would it be large enough to 
be interesting, whether in theory or in practice?
2. Can there be any feasible way to demonstrate the proposed effect by 
measurements?


The first thing one should do is to find a way to estimate the magnitude 
of the effect so it can be compared with some of those other phenomena 
(non-constant gravity, etc) to see if it's worth doing a full 
computation at all, or even spending any more time on the matter.  There 
is likely to be a way to make such an estimate without needing to resort 
to extremely high precision - you would only need to get within perhaps 
an order of magnitude.  Your real task, then, is to find that way.


For example, you would probably be able to estimate the precision needed 
without actually doing the calculation.  That in itself might turn out 
to enough.




--
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Peter J. Holzer]

On 2022-12-19 09:25:17 +1100, Chris Angelico wrote:
> On Mon, 19 Dec 2022 at 07:57, Stefan Ram  wrote:
> > G = Decimal( 6.6743015E-11 )
> > r = Decimal( 6.371E6 )
> > M = Decimal( 5.9722E24 )
> 
> What's the point of using Decimal if you start with nothing more than
> float accuracy?

Right. He also interpreted the notation "6.67430(15)E-11" wrong. The
digits in parentheses represent the uncertainty in the same number of
last digits. So "6.67430(15)E-11" means "something between 6.67430E-11 -
0.00015E-11 and 6.67430E-11 + 0.00015E-11". The r value has only a
precision of 1 km and I'm not sure how accurate the mass is. Let's just
assume (for the sake of the argument) that these are actually accurate in
all given digits.

So G is between 6.67415E-11 and 6.67445E-11, r is between 6.3705E6 and
6.3715E6 and M is between 5.97215E24 and 5.97225E24. If we compute the
time for those deviations you will find that the differences are many
orders of magnitude greater than the effect you wanted to show. And that
still ignores the fact that a vacuum won't be perfect (and collisions
with a few stray atoms might have a similarly tiny effect), that gravity
isn't constant while the weight falls (it's getting closer to the center
of the earth and it's moving past other masses on its way) that Newton's
law is only an approximation, etc. So while the effect is (almost
certainly) real, the numbers are garbage.

I think there's a basic numeracy problem here. This is unfortunately all
too common, even among scientists. The OP apparently rounded their
numbers to 8 significant digits (thereby introducing an error of about
1E-8) and then insisted that the additional error of 1E-15 introduced by
the decimal to float conversion was unacceptable, showing IMHO a
fundamental misunderstanding of the numbers they are working with.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Cameron Simpson]

On 19Dec2022 08:53, Cameron Simpson  wrote:
I'm no expert on floating point coding for precision, but I believe 
that trying to work with values "close together" in magnitude is 
important because values of different scales inherently convert one of 
them to the other scale (i.e. similar sized exponent part) with 
corresponding loss of precision in the mantissa part. That may require 
you to form your calcutations carefully.


This depends on the operation. With addition/subtraction you'd see this 
directly. With multiplication etc it isn't really the case. But there 
are both more and less effective ways to arrange your floating point 
math in terms of preserving what precision you have.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Greg Ewing]

On 19/12/22 6:35 am, Paul St George wrote:

So I am working on a physics paper with a colleague. We have a theory about 
Newtons Cradle.

We want to illustrate the paper with animations.

Because there is a problem, I am investigating in all areas. ... I would like 
to be in control of or fully aware of what goes on under the bonnet.


When you convert a string to a float, you're already getting the closest
possible value in binary floating point.

For things like physics simulations, you need to design your algorithms
so that they're tolerant of small inaccuracies in the representation of
your numbers. If those are causing you problems, it sounds like there
is some kind of numerical instability in your algorithm that needs to
be addressed.

It's also possible that there is just a bug somewhere in your code, and
the problem really has nothing to do with floating point inaccuracies.

If you can post some code we might be able to help you further.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Chris Angelico]

On Mon, 19 Dec 2022 at 07:57, Stefan Ram  wrote:
> G = Decimal( 6.6743015E-11 )
> r = Decimal( 6.371E6 )
> M = Decimal( 5.9722E24 )

What's the point of using Decimal if you start with nothing more than
float accuracy?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Cameron Simpson]

On 18Dec2022 18:35, Paul St George  wrote:

So I am working on a physics paper with a colleague. We have a theory about 
Newtons Cradle. We answer the question why when you lift and drop balls 1 and 
2, balls 4 and 5 rise up. I could say more, but ... (if you are interested 
please write to me).

We want to illustrate the paper with animations. The theory includes distortion 
of the balls and this distortion is very very small. So, I am sent data with 
locations and dimensions to 13 decimal places. Something strange is happening 
with the animations: the balls are not moving smoothly. I do not know (yet) 
where the problem lies so it is difficult to provide a clear narrative.

Because there is a problem, I am investigating in all areas. This brings me to 
the question I asked here. I am not expecting six decimal places or three 
decimal places to be as accurate as thirteen decimal places, but I would like 
to be in control of or fully aware of what goes on under the bonnet.


First the short take: your machine pobably is quite precise, and float 
is far more performant that the other numeric types available. Your 
source data seem to have more round off than the rounding in a float.


Under the bonnet:

A Python float is effectively a base-2 value in scientific notation.  
Internally it has a base-2 mantissa and base-2 exponent. This page:

https://docs.python.org/3/library/stdtypes.html#typesnumeric
says that CPython's float uses C's "double" floating point type
(you are almost certainly using the CPython implementation) and thus 
you're using the machine's floating point implemenetation.


I believe that almost all modern CPUs implement IEEE 754 floating point:
https://en.wikipedia.org/wiki/IEEE_754

Because they're base 2, various values in other bases will not be 
precisely representable as a float. For example, 1/3 (which you will 
know is _also_ not representable precisely as a base-10 value such as 
0.333).


You can get specifics of your Python's floating point from 
`sys.float_Info`, i.e:


from sys import float_info

The look at float_info.epsilon etc. Details:
https://docs.python.org/3/library/sys.html#sys.float_info

Here's my machine:

Python 3.10.6 (main, Aug 11 2022, 13:47:18) [Clang 12.0.0 
(clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more 
information.

>>> from sys import float_info
>>> float_info
sys.float_info(max=1.7976931348623157e+308, max_exp=1024, 
max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021, 
min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16, 
radix=2, rounds=1)


Values of note: mant_dig=53 (53 base-2 bits), dig=15 (15 decimal digits 
of precision).


You might want to look at sys.float_repr_style here:
https://docs.python.org/3/library/sys.html#sys.float_repr_style
which affects how Python writes floats out. In particular this text:

 If the string has value 'short' then for a finite float x, repr(x) 
 aims to produce a short string with the property that 
 float(repr(x)) == x. This is the usual behaviour in Python 3.1 and 
 later.


Again, on my machine:

>>> 64550.727
64550.727
>>> 64550.728
64550.728
>>> 64550.72701
64550.72701
>>> 64550.7270101
64550.7270101
>>> 64550.727010101
64550.727010101
>>> 64550.72701010101
64550.72701010101
>>> 64550.7270101010101
64550.72701010101
>>> 64550.727010101010101
64550.72701010101
>>> 64550.72701010101010101
64550.72701010101


On 17 Dec 2022, at 16:54:05 EST 2022, Thomas Passin wrote:

On 12/17/2022 3:45 PM, Paul St George wrote:

Thanks to all!
It was the rounding rounding error that I needed to avoid (as Peter J. Holzer 
suggested). The use of decimal solved it and just in time. I was about to 
truncate the number, get each of the characters from the string mantissa, and 
then do something like this:

64550.727

64550 + (7 * 0.1) + (2 * 0.01) + (7 * 0.001)

Now I do not need to!


Good, because if you do that using floats it will be less precise than 
float(64550.727). (Which I see Alan has already stated.)


Your source file contains strings like "64550.727". They look to already 
be less than 13 digits of precision as written i.e. some round off 
already took place when that file was written. Do you know the precision 
of the source data?


I suspect that rather than chasing a "perfect" representation of your 
source data, which is already rounded off, you:

- see if the source values can be obtained more precisely
- figure out which operations in your simulation contribute to the 
  motion roughness you see


I'm no expert on floating point coding for precision, but I believe that 
trying to work with values "close together" in magnitude is important 
because values of different scales inherently convert one of them to the 
other scale (i.e. similar sized exponent part) with corresponding loss 
of precision in the mantissa part. 

Re: String to Float, without introducing errors

[dn]

On 18/12/2022 10.55, Stefan Ram wrote:

Grant Edwards  writes:

Yes, fixed point (or decimal) is a better fit for what he's doing. but
I suspect that floating point would be a better fit for the problem
he's trying to solve.


   I'd like to predict that within the next ten posts in this
   thread someone will mention "What Every Computer Scientist
   Should Know About Floating-Point Arithmetic".



Thank you for doing-so.

More specific: https://docs.python.org/3/tutorial/floatingpoint.html


Perhaps in the rush to 'answer', the joke is on 'us' - that we might 
first need to more carefully-understand the OP's use-case, requirements, 
and constraints?



The joke sours when remembering that this 'mystery' generates frequent 
questions 'here' (and on other Python fora) - not as many as 'why don't 
I see a pretty-GUI when I fire-up Python on MS-Windows?' but it is a 
more sophisticated realisation and deserves a detailed response (such as 
the thought-provoking illustration-code appearing today).



Is it a consequence of Python lowering 'the barrier to entry'? Good 
thing? Bad thing?


(we first started noticing this sort of issue in our (non-Python) MOOCs, 
several pre-COVID years ago: when we first started, the trainee was 
typically recent post-grad; whereas today we enrol folk with a much 
wider range of ages and backgrounds. It is likely that more dev.work has 
gone into 'the bottom end', than into new/higher sophistications - even 
given IT's rate-of-change!)


--
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

String to Float, without introducing errors

[Paul St George]

So I am working on a physics paper with a colleague. We have a theory about 
Newtons Cradle. We answer the question why when you lift and drop balls 1 and 
2, balls 4 and 5 rise up. I could say more, but ... (if you are interested 
please write to me).

We want to illustrate the paper with animations. The theory includes distortion 
of the balls and this distortion is very very small. So, I am sent data with 
locations and dimensions to 13 decimal places. Something strange is happening 
with the animations: the balls are not moving smoothly. I do not know (yet) 
where the problem lies so it is difficult to provide a clear narrative.

Because there is a problem, I am investigating in all areas. This brings me to 
the question I asked here. I am not expecting six decimal places or three 
decimal places to be as accurate as thirteen decimal places, but I would like 
to be in control of or fully aware of what goes on under the bonnet.







 
>> On 17 Dec 2022, at 16:54:05 EST 2022, Thomas Passin wrote:
On 12/17/2022 3:45 PM, Paul St George wrote:
> Thanks to all!
> It was the rounding rounding error that I needed to avoid (as Peter J. Holzer 
> suggested). The use of decimal solved it and just in time. I was about to 
> truncate the number, get each of the characters from the string mantissa, and 
> then do something like this:
> 
> 64550.727
> 
> 64550 + (7 * 0.1) + (2 * 0.01) + (7 * 0.001)
> 
> Now I do not need to!

And that approach would not have helped you, because each of those 
calculations would be done as floating point, and you wouldn't have 
gotten any more precision (and maybe less) than simply doing 
float('64550.727').

Here is a small but interesting discussion thread about float vs Decimal:

https://stackoverflow.com/questions/32053647/comparing-python-decimals-created-from-float-and-string

Would you mind telling us why that degree of precision (that is, decimal 
vs float) matters for your problem?


>> On 17 Dec 2022, at 13:11, Alan Gauld > > wrote:
>>
>> On 17/12/2022 11:51, Paul St George wrote:
>>> I have a large/long array of numbers in an external file. The numbers look 
>>> like this:
>>>
>>> -64550.727
>>> -64511.489
>>> -64393.637
>>> -64196.763
>>> -63920.2
>>
>>> When I bring the numbers into my code, they are Strings. To use the
>>> numbers in my code, I want to change the Strings to Float type
>>> because the code will not work with Strings but I do not want
>>> to change the numbers in any other way.
>>
>> That may be impossible. Float type is not exact and the conversion
>> will be the closest binary representation of your decimal number.
>> It will be very close but it may be slightly different when you
>> print it, for example. (You can usually deal with that by using
>> string formatting features.)
>>
>> Another option is to use the decimal numeric type. That has other
>> compromises associated with it but, if retaining absolute decimal
>> accuracy is your primary goal, it might suit you better.
>>
>>
>> -- 
>> Alan G
>> Author of the Learn to Program web site
>> http://www.alan-g.me.uk/
>> http://www.amazon.com/author/alan_gauld
>> Follow my photo-blog on Flickr at:
>> http://www.flickr.com/photos/alangauldphotos
>>
>>
> 









-- 
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[rbowman]

On Sun, 18 Dec 2022 11:14:28 -0500, Dennis Lee Bieber wrote:

> .. And maybe lament the days when a 3-digit result was acceptable in
> math class -- being the typical capability in reading a standard (10"
> scale) slide rule.

Arguably more thought was given to what those three digits meant in the 
real world. For example, is calculating a latitude to 6 decimal places 
meaningful when the data was gathered by a GPS receiver with 5m accuracy? 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Thomas Passin]

Thanks for filling us in!  I wouldn't think the animations themselves 
would need such precision, though perhaps the calculations of the forces 
and motions do.  One way to check might be to perturb the initial 
conditions a bit and see if the changes in the motions seem to be 
correspondingly small.  That might help you work out if the problem is 
with the physics solution or the animation.  It would be easy to get 
some numerical instability in the calculations, for example if you are 
inverting matrices.


On 12/18/2022 2:00 PM, Paul St George wrote:

So I am working on a physics paper with a colleague. We have a theory about 
Newtons Cradle. We answer the question why when you lift and drop balls 1 and 
2, balls 4 and 5 rise up. I could say more, but ... (if you are interested 
please write to me).

We want to illustrate the paper with animations. The theory includes distortion 
of the balls and this distortion is very very small. So, I am sent data with 
locations and dimensions to 13 decimal places. Something strange is happening 
with the animations: the balls are not moving smoothly. I do not know (yet) 
where the problem lies so it is difficult to provide a clear narrative.

Because there is a problem, I am investigating in all areas. This brings me to 
the question I asked here. I am not expecting six decimal places or three 
decimal places to be as accurate as thirteen decimal places, but I would like 
to be in control of or fully aware of what goes on under the bonnet.

Here is a picture:
https://paulstgeorge.com/newton/cyclography.html
Thanks,
Paul



  

On 17 Dec 2022, at 16:54:05 EST 2022, Thomas Passin wrote:

On 12/17/2022 3:45 PM, Paul St George wrote:

Thanks to all!
It was the rounding rounding error that I needed to avoid (as Peter J. Holzer 
suggested). The use of decimal solved it and just in time. I was about to 
truncate the number, get each of the characters from the string mantissa, and 
then do something like this:

64550.727

64550 + (7 * 0.1) + (2 * 0.01) + (7 * 0.001)

Now I do not need to!


And that approach would not have helped you, because each of those
calculations would be done as floating point, and you wouldn't have
gotten any more precision (and maybe less) than simply doing
float('64550.727').

Here is a small but interesting discussion thread about float vs Decimal:

https://stackoverflow.com/questions/32053647/comparing-python-decimals-created-from-float-and-string

Would you mind telling us why that degree of precision (that is, decimal
vs float) matters for your problem?



On 17 Dec 2022, at 13:11, Alan Gauld https://mail.python.org/mailman/listinfo/python-list>> wrote:

On 17/12/2022 11:51, Paul St George wrote:

I have a large/long array of numbers in an external file. The numbers look like 
this:

-64550.727
-64511.489
-64393.637
-64196.763
-63920.2



When I bring the numbers into my code, they are Strings. To use the
numbers in my code, I want to change the Strings to Float type
because the code will not work with Strings but I do not want
to change the numbers in any other way.


That may be impossible. Float type is not exact and the conversion
will be the closest binary representation of your decimal number.
It will be very close but it may be slightly different when you
print it, for example. (You can usually deal with that by using
string formatting features.)

Another option is to use the decimal numeric type. That has other
compromises associated with it but, if retaining absolute decimal
accuracy is your primary goal, it might suit you better.


--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
















--
https://mail.python.org/mailman/listinfo/python-list

String to Float, without introducing errors

[Paul St George]

So I am working on a physics paper with a colleague. We have a theory about 
Newtons Cradle. We answer the question why when you lift and drop balls 1 and 
2, balls 4 and 5 rise up. I could say more, but ... (if you are interested 
please write to me).

We want to illustrate the paper with animations. The theory includes distortion 
of the balls and this distortion is very very small. So, I am sent data with 
locations and dimensions to 13 decimal places. Something strange is happening 
with the animations: the balls are not moving smoothly. I do not know (yet) 
where the problem lies so it is difficult to provide a clear narrative.

Because there is a problem, I am investigating in all areas. This brings me to 
the question I asked here. I am not expecting six decimal places or three 
decimal places to be as accurate as thirteen decimal places, but I would like 
to be in control of or fully aware of what goes on under the bonnet.

Here is a picture:
https://paulstgeorge.com/newton/cyclography.html
Thanks,
Paul



 
>> On 17 Dec 2022, at 16:54:05 EST 2022, Thomas Passin wrote:
On 12/17/2022 3:45 PM, Paul St George wrote:
> Thanks to all!
> It was the rounding rounding error that I needed to avoid (as Peter J. Holzer 
> suggested). The use of decimal solved it and just in time. I was about to 
> truncate the number, get each of the characters from the string mantissa, and 
> then do something like this:
> 
> 64550.727
> 
> 64550 + (7 * 0.1) + (2 * 0.01) + (7 * 0.001)
> 
> Now I do not need to!

And that approach would not have helped you, because each of those 
calculations would be done as floating point, and you wouldn't have 
gotten any more precision (and maybe less) than simply doing 
float('64550.727').

Here is a small but interesting discussion thread about float vs Decimal:

https://stackoverflow.com/questions/32053647/comparing-python-decimals-created-from-float-and-string

Would you mind telling us why that degree of precision (that is, decimal 
vs float) matters for your problem?


>> On 17 Dec 2022, at 13:11, Alan Gauld > > wrote:
>>
>> On 17/12/2022 11:51, Paul St George wrote:
>>> I have a large/long array of numbers in an external file. The numbers look 
>>> like this:
>>>
>>> -64550.727
>>> -64511.489
>>> -64393.637
>>> -64196.763
>>> -63920.2
>>
>>> When I bring the numbers into my code, they are Strings. To use the
>>> numbers in my code, I want to change the Strings to Float type
>>> because the code will not work with Strings but I do not want
>>> to change the numbers in any other way.
>>
>> That may be impossible. Float type is not exact and the conversion
>> will be the closest binary representation of your decimal number.
>> It will be very close but it may be slightly different when you
>> print it, for example. (You can usually deal with that by using
>> string formatting features.)
>>
>> Another option is to use the decimal numeric type. That has other
>> compromises associated with it but, if retaining absolute decimal
>> accuracy is your primary goal, it might suit you better.
>>
>>
>> -- 
>> Alan G
>> Author of the Learn to Program web site
>> http://www.alan-g.me.uk/
>> http://www.amazon.com/author/alan_gauld
>> Follow my photo-blog on Flickr at:
>> http://www.flickr.com/photos/alangauldphotos
>>
>>
> 









-- 
https://mail.python.org/mailman/listinfo/python-list

RE: String to Float, without introducing errors

[avi.e.gross]

As often seems to happen, someone asks something that may not be fully clear 
and others chime in and extend the question.

Was the original question how to read in a ingle column of numbers from a file 
that are all numeric and NOT integers and be able to use them?

If so, the answer was quite trivial using the conversion function of your 
choice. Handling errors or what to do with something like a blank or NA are 
nice ideas if asked about.

My answer would be to ask if this was an assignment where they are EXPECTED to 
do things a certain way to master a concept, or part of a serious attempt to 
get things done.

For the latter case, it may make plenty of sense considering a single column of 
text as just a special case for the kind of multi-column files often read in 
from formatted data files and use some functionality from add-on modules like 
numpy or pandas that also allow you to deal with many other concerns.

Note such utilities also often make a decent guess on what data type a column 
should be turned into and it is possible they may occasionally decide based on 
the data that the contents all HAPPEN to be integer or there is at least one 
that makes it choose character. So any such use may well require the subsequent 
use of functions that check the dtype and, if needed, do an explicit conversion 
to what you really really really want.



-Original Message-
From: Python-list  On 
Behalf Of Mats Wichmann
Sent: Saturday, December 17, 2022 1:42 PM
To: python-list@python.org
Subject: Re: String to Float, without introducing errors

On 12/17/22 07:15, Thomas Passin wrote:
> You have strings, and you want to end up with numbers.  The numbers 
> are not integers.  Other responders have gone directly to whether you 
> should use float or decimal as the conversion, but that is a secondary matter.
> 
> If you have integers, convert with
> 
> integer = int(number_string)
>> -64550.727

they pretty clearly aren't integers:

>> -64511.489
>> -64393.637
>> -64196.763
>> -63920.2
>> -63563.037
>> -63124.156
>> -62602.254
>> -61995.895
>> -61303.548
>> -60523.651
>> -59654.66
--
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Chris Angelico]

On Sun, 18 Dec 2022 at 09:46, Stefan Ram  wrote:
>
> Grant Edwards  writes:
> >Yes, fixed point (or decimal) is a better fit for what he's doing. but
> >I suspect that floating point would be a better fit for the problem
> >he's trying to solve.
>
>   I'd like to predict that within the next ten posts in this
>   thread someone will mention "What Every Computer Scientist
>   Should Know About Floating-Point Arithmetic".
>
> |>>> 0.1 + 0.2 - 0.3
> |5.551115123125783e-17
>

Looks like someone just did.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[dn]

On 18/12/2022 01.39, Peter J. Holzer wrote:

On 2022-12-17 12:51:17 +0100, Paul St George wrote:

I have a large/long array of numbers in an external file. The numbers
look like this:

-64550.727
-64511.489
-64393.637

[...]


When I bring the numbers into my code, they are Strings. To use the
numbers in my code, I want to change the Strings to Float type because
the code will not work with Strings but I do not want to change the
numbers in any other way.



s = "-64550.727"
f = float(s)
f

-64550.727

type(f)



(Contrary to the other people posting in this thread I don't think float
is the wrong type for the job. It might be, but you haven't given enough
details to tell whether the inevitable rounding error matters or not. In
my experience in almost all cases where people think it matters it
really doesn't.)


Agreed: (ultimately) insufficient information-provided.
(but that probably doesn't matter either - as the OP seems to have come 
to a decision)



Agreed: probably doesn't matter.


'The world' agrees with both, having decided that Numerical Analysis is 
no-longer a necessary ComSc study.


In the ?good, old, days Numerical Analysis included contemplation of the 
difficulties and differences between "precision" and "accuracy". Thus, 
the highly accurate calculation of less-than precise numbers - or was it 
precise values subject to less than accurate computation?

(rhetorical!)

Sort of like giving highly-accurate answers to a less-than precise 
(complete) question, by presuming can ignore the latter.


--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Thomas Passin]

On 12/17/2022 3:45 PM, Paul St George wrote:

Thanks to all!
It was the rounding rounding error that I needed to avoid (as Peter J. Holzer 
suggested). The use of decimal solved it and just in time. I was about to 
truncate the number, get each of the characters from the string mantissa, and 
then do something like this:

64550.727

64550 + (7 * 0.1) + (2 * 0.01) + (7 * 0.001)

Now I do not need to!


And that approach would not have helped you, because each of those 
calculations would be done as floating point, and you wouldn't have 
gotten any more precision (and maybe less) than simply doing 
float('64550.727').


Here is a small but interesting discussion thread about float vs Decimal:

https://stackoverflow.com/questions/32053647/comparing-python-decimals-created-from-float-and-string

Would you mind telling us why that degree of precision (that is, decimal 
vs float) matters for your problem?




On 17 Dec 2022, at 13:11, Alan Gauld  wrote:

On 17/12/2022 11:51, Paul St George wrote:

I have a large/long array of numbers in an external file. The numbers look like 
this:

-64550.727
-64511.489
-64393.637
-64196.763
-63920.2



When I bring the numbers into my code, they are Strings. To use the
numbers in my code, I want to change the Strings to Float type
because the code will not work with Strings but I do not want
to change the numbers in any other way.


That may be impossible. Float type is not exact and the conversion
will be the closest binary representation of your decimal number.
It will be very close but it may be slightly different when you
print it, for example. (You can usually deal with that by using
string formatting features.)

Another option is to use the decimal numeric type. That has other
compromises associated with it but, if retaining absolute decimal
accuracy is your primary goal, it might suit you better.


--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos






--
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Peter J. Holzer]

On 2022-12-17 21:45:06 +0100, Paul St George wrote:
> It was the rounding rounding error that I needed to avoid (as Peter J.
> Holzer suggested). The use of decimal solved it and just in time. I
> was about to truncate the number, get each of the characters from the
> string mantissa, and then do something like this:
> 
> 64550.727
> 
> 64550 + (7 * 0.1) + (2 * 0.01) + (7 * 0.001)

That wouldn't have helped. In fact it would have made matters worse
because instead of a single rounding operation you now have nine!

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Chris Angelico]

On Sun, 18 Dec 2022 at 08:22, Grant Edwards  wrote:
>
> On 2022-12-17, Chris Angelico  wrote:
>
> >> It was the rounding rounding error that I needed to avoid (as Peter
> >> J. Holzer suggested). The use of decimal solved it and just in
> >> time. I was about to truncate the number, get each of the
> >> characters from the string mantissa, and then do something like
> >> this:
> >>
> >> 64550.727
> >>
> >> 64550 + (7 * 0.1) + (2 * 0.01) + (7 * 0.001)
> >>
> >> Now I do not need to!
> >
> > It sounds like fixed-point arithmetic might be a better fit for what
> > you're doing.
>
> Yes, fixed point (or decimal) is a better fit for what he's doing. but
> I suspect that floating point would be a better fit for the problem
> he's trying to solve.
>

Hard to judge, given how little info we have on the actual problem.
Could go either way.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Grant Edwards]

On 2022-12-17, Chris Angelico  wrote:

>> It was the rounding rounding error that I needed to avoid (as Peter
>> J. Holzer suggested). The use of decimal solved it and just in
>> time. I was about to truncate the number, get each of the
>> characters from the string mantissa, and then do something like
>> this:
>>
>> 64550.727
>>
>> 64550 + (7 * 0.1) + (2 * 0.01) + (7 * 0.001)
>>
>> Now I do not need to!
>
> It sounds like fixed-point arithmetic might be a better fit for what
> you're doing.

Yes, fixed point (or decimal) is a better fit for what he's doing. but
I suspect that floating point would be a better fit for the problem
he's trying to solve.

--
Grant



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Chris Angelico]

On Sun, 18 Dec 2022 at 07:46, Paul St George  wrote:
>
> Thanks to all!
> It was the rounding rounding error that I needed to avoid (as Peter J. Holzer 
> suggested). The use of decimal solved it and just in time. I was about to 
> truncate the number, get each of the characters from the string mantissa, and 
> then do something like this:
>
> 64550.727
>
> 64550 + (7 * 0.1) + (2 * 0.01) + (7 * 0.001)
>
> Now I do not need to!

It sounds like fixed-point arithmetic might be a better fit for what
you're doing.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Paul St George]

Thanks to all!
It was the rounding rounding error that I needed to avoid (as Peter J. Holzer 
suggested). The use of decimal solved it and just in time. I was about to 
truncate the number, get each of the characters from the string mantissa, and 
then do something like this:

64550.727

64550 + (7 * 0.1) + (2 * 0.01) + (7 * 0.001)

Now I do not need to!





> On 17 Dec 2022, at 13:11, Alan Gauld  wrote:
> 
> On 17/12/2022 11:51, Paul St George wrote:
>> I have a large/long array of numbers in an external file. The numbers look 
>> like this:
>> 
>> -64550.727
>> -64511.489
>> -64393.637
>> -64196.763
>> -63920.2
> 
>> When I bring the numbers into my code, they are Strings. To use the 
>> numbers in my code, I want to change the Strings to Float type 
>> because the code will not work with Strings but I do not want 
>> to change the numbers in any other way.
> 
> That may be impossible. Float type is not exact and the conversion
> will be the closest binary representation of your decimal number.
> It will be very close but it may be slightly different when you
> print it, for example. (You can usually deal with that by using
> string formatting features.)
> 
> Another option is to use the decimal numeric type. That has other
> compromises associated with it but, if retaining absolute decimal
> accuracy is your primary goal, it might suit you better.
> 
> 
> -- 
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
> 
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Thomas Passin]

On 12/17/2022 1:41 PM, Mats Wichmann wrote:

On 12/17/22 07:15, Thomas Passin wrote:
You have strings, and you want to end up with numbers.  The numbers 
are not integers.  Other responders have gone directly to whether you 
should use float or decimal as the conversion, but that is a secondary 
matter.


If you have integers, convert with

integer = int(number_string)

-64550.727


they pretty clearly aren't integers:


Of course they aren't.  That's why I gave the line with float() too. 
It's useful to see that there is a basic pattern here:  Given a string, 
expect to need to convert it to what you actually want, int, float, 
decimal, whatever.



-64511.489
-64393.637
-64196.763
-63920.2
-63563.037
-63124.156
-62602.254
-61995.895
-61303.548
-60523.651
-59654.66


--
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Mats Wichmann]

On 12/17/22 07:15, Thomas Passin wrote:
You have strings, and you want to end up with numbers.  The numbers are 
not integers.  Other responders have gone directly to whether you should 
use float or decimal as the conversion, but that is a secondary matter.


If you have integers, convert with

integer = int(number_string)

-64550.727


they pretty clearly aren't integers:


-64511.489
-64393.637
-64196.763
-63920.2
-63563.037
-63124.156
-62602.254
-61995.895
-61303.548
-60523.651
-59654.66

--
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Thomas Passin]

You have strings, and you want to end up with numbers.  The numbers are 
not integers.  Other responders have gone directly to whether you should 
use float or decimal as the conversion, but that is a secondary matter.


If you have integers, convert with

integer = int(number_string)

If you don't have integers, convert with

number = float(number_string)

If the number is not an integer and you need to be extremely precise 
with the fractional part - for example, if you need to keep exact track 
of exact amounts of money - you can use decimal instead of float.  But 
in this example it seems unlikely that you need that.


The thing to be aware of that hasn't been mentioned so far is that the 
conversion might cause an exception if there is something wrong with one 
of the number strings.  If you don't handle it, your program will quit 
running when it hits the error.  That might not matter to you.  If it 
matters, you can handle the exception in the program, but first you will 
need to decide what to do about such an error: should that number be 
omitted, should it be replaced with a placeholder, should it be replaced 
with float("NaN"), or what else?


In your case here, I'd suggest not handling the error until you get the 
basics of your program working.  Then if you need to, figure out how you 
want to handle exceptions.



On 12/17/2022 6:51 AM, Paul St George wrote:

I have a large/long array of numbers in an external file. The numbers look like 
this:

-64550.727
-64511.489
-64393.637
-64196.763
-63920.2
-63563.037
-63124.156
-62602.254
-61995.895
-61303.548
-60523.651
-59654.66
...

When I bring the numbers into my code, they are Strings. To use the numbers in 
my code, I want to change the Strings to Float type because the code will not 
work with Strings but I do not want to change the numbers in any other way.

So, I want my Strings (above) to be these numbers.

-64550.727
-64511.489
-64393.637
-64196.763
-63920.2
-63563.037
-63124.156
-62602.254
-61995.895
-61303.548
-60523.651
-59654.66
...

Please help!












--
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Peter J. Holzer]

On 2022-12-17 12:51:17 +0100, Paul St George wrote:
> I have a large/long array of numbers in an external file. The numbers
> look like this:
> 
> -64550.727
> -64511.489
> -64393.637
[...]
> 
> When I bring the numbers into my code, they are Strings. To use the
> numbers in my code, I want to change the Strings to Float type because
> the code will not work with Strings but I do not want to change the
> numbers in any other way.

>>> s = "-64550.727"
>>> f = float(s)
>>> f
-64550.727
>>> type(f)


(Contrary to the other people posting in this thread I don't think float
is the wrong type for the job. It might be, but you haven't given enough
details to tell whether the inevitable rounding error matters or not. In
my experience in almost all cases where people think it matters it
really doesn't.)

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Alan Gauld]

On 17/12/2022 11:51, Paul St George wrote:
> I have a large/long array of numbers in an external file. The numbers look 
> like this:
> 
> -64550.727
> -64511.489
> -64393.637
> -64196.763
> -63920.2

> When I bring the numbers into my code, they are Strings. To use the 
> numbers in my code, I want to change the Strings to Float type 
> because the code will not work with Strings but I do not want 
> to change the numbers in any other way.

That may be impossible. Float type is not exact and the conversion
will be the closest binary representation of your decimal number.
It will be very close but it may be slightly different when you
print it, for example. (You can usually deal with that by using
string formatting features.)

Another option is to use the decimal numeric type. That has other
compromises associated with it but, if retaining absolute decimal
accuracy is your primary goal, it might suit you better.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: String to Float, without introducing errors

[Weatherby,Gerard]

https://docs.python.org/3/library/decimal.html

Get Outlook for iOS<https://aka.ms/o0ukef>

From: Python-list  on 
behalf of Paul St George 
Sent: Saturday, December 17, 2022 6:51:17 AM
To: python-list@python.org 
Subject: String to Float, without introducing errors

*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

I have a large/long array of numbers in an external file. The numbers look like 
this:

-64550.727
-64511.489
-64393.637
-64196.763
-63920.2
-63563.037
-63124.156
-62602.254
-61995.895
-61303.548
-60523.651
-59654.66
...

When I bring the numbers into my code, they are Strings. To use the numbers in 
my code, I want to change the Strings to Float type because the code will not 
work with Strings but I do not want to change the numbers in any other way.

So, I want my Strings (above) to be these numbers.

-64550.727
-64511.489
-64393.637
-64196.763
-63920.2
-63563.037
-63124.156
-62602.254
-61995.895
-61303.548
-60523.651
-59654.66
...

Please help!










--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!guE_zPsjxMW4k6nHIdOqZbrt8SdjUC9GELXgSHatARIr2PrAYr6tXCmixkZGNocjsf9SKLduQFjZjM7tOeaQ$
-- 
https://mail.python.org/mailman/listinfo/python-list

String to Float, without introducing errors

[Paul St George]

I have a large/long array of numbers in an external file. The numbers look like 
this:

-64550.727
-64511.489
-64393.637
-64196.763
-63920.2
-63563.037
-63124.156
-62602.254
-61995.895
-61303.548
-60523.651
-59654.66
...

When I bring the numbers into my code, they are Strings. To use the numbers in 
my code, I want to change the Strings to Float type because the code will not 
work with Strings but I do not want to change the numbers in any other way.

So, I want my Strings (above) to be these numbers.

-64550.727
-64511.489
-64393.637
-64196.763
-63920.2
-63563.037
-63124.156
-62602.254
-61995.895
-61303.548
-60523.651
-59654.66
...

Please help!










-- 
https://mail.python.org/mailman/listinfo/python-list