Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Ben Bacarisse]

Chris Kaynor  writes:

> On Thu, Jul 21, 2016 at 4:54 PM, Ben Bacarisse  wrote:
>
>> Steven D'Aprano  writes:
>> 
>> > Or you might be using a language like Javascript, which intentionally has
>> > only floats for numbers. That's okay, you can still perform exact integer
>> > arithmetic, so long as you stay within the bounds of ±2**16.
>>
>> Small point: it's 2**52.
>>
>
> If you really want to be picky, it is 2**53, inclusive:

Yes, I mis-typed.  Typical, I suppose, for a correction!

To add something useful, there are properties Number.MAX_SAFE_INTEGER
and Number.MIN_SAFE_INTEGER which are 2**53 - 1 and -2**53 + 1.  These
are the largest (and smallest) integers such that i and i+1 (or i and
i-1) are exactly representable.


-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Chris Kaynor]

On Thu, Jul 21, 2016 at 4:54 PM, Ben Bacarisse  wrote:

> Steven D'Aprano  writes:
> 
> > Or you might be using a language like Javascript, which intentionally has
> > only floats for numbers. That's okay, you can still perform exact integer
> > arithmetic, so long as you stay within the bounds of ±2**16.
>
> Small point: it's 2**52.
>

If you really want to be picky, it is 2**53, inclusive:

>>> 2**53-2.0

9007199254740990.0

>>> 2**53-1.0

9007199254740991.0

>>> 2**53+0.0 # Can no longer store odd numbers, but 2**53 is even so it
can still be stored.

9007199254740992.0

>>> 2**53+1.0

9007199254740992.0

>>> 2**53+2.0
9007199254740994.0

This works as there is an implied one bit in the field for floats, giving
53 bits of storage despite only having 52 bits of storage. As the sign is
stored in a separate bit, the same limit applies to both positive and
negative numbers.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Ben Bacarisse]

Steven D'Aprano  writes:

> Or you might be using a language like Javascript, which intentionally has
> only floats for numbers. That's okay, you can still perform exact integer
> arithmetic, so long as you stay within the bounds of ±2**16.

Small point: it's 2**52.


-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Steven D'Aprano]

On Thu, 21 Jul 2016 11:14 pm, Rustom Mody wrote:

> On Thursday, July 21, 2016 at 12:04:35 PM UTC+5:30, Steven D'Aprano wrote:
>> On Thursday 21 July 2016 15:28, Rustom Mody wrote:
>> > BTW APL whose main domain of application is scientific chooses to
>> > enshrine this —equality is ε-neighborhood checking not exact equality
>> > checking — into its builtin ‘==’
>> 
>> I have a lot of respect for Ken Iverson, and a lot of admiration for
>> language designers willing to experiment with alternate paradigms.
> 
> This choice has significant(!!) costs: Fuzzy equality is not transitive:
> One can get
> a = b ∧ b = c ∧ a ≠ c
>> 
>> But keeping in mind that in APL, if you set ⎕ct to 0 you get an exact
>> comparison, can you find any quotes from Iverson saying that you should
>> *never* perform exact equality comparisons?
> 
> There you go with your strawmen!
> Remember it was you (and Chris) who expressed extreme positions:
> “Pernicious myth” “FUD” etc


And YOU'RE the one who is raising APL as an argument *against* my
characterisation.

Do you think that Iverson would agree with the conventional wisdom that we
should NEVER test floats for exact equality?

Do you know of ANY expert in numeric computation who will agree with the
conventional wisdom? If so, who?

I'll admit that I've stolen my description of this rule as "superstition"
from perhaps the world's foremost authority on numeric computation,
Professor William Kahan. (See the forward to "Apple Numerics Manual, Second
Edition, 1988.) You don't like my use of the term "pernicious"? In my
opinion, any conventional wisdom which keeps people *more* rather than
*less* ignorant is pernicious. Anything which discourages them from testing
their numeric functions to the full precision possible is pernicious.
Anything which encourages the idea that numeric computation using floats is
non-deterministic is pernicious.

But if you don't agree, feel free to dismiss the word as mere hyperbola and
MOVE ON. You don't have to nit pick about every word I use.

The bottom line is, while it is often true that using exact equality is
going to surprise people, the conventional wisdom to NEVER do so is both
(1) factually wrong, there are plenty of examples where one can and should
use exact equality, and (2) rather useless, as the conventional wisdom
gives no clue as to the practicalities of what to replace it with.




-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Rustom Mody]

On Thursday, July 21, 2016 at 12:04:35 PM UTC+5:30, Steven D'Aprano wrote:
> On Thursday 21 July 2016 15:28, Rustom Mody wrote:
> > BTW APL whose main domain of application is scientific chooses to enshrine
> > this —equality is ε-neighborhood checking not exact equality checking — into
> > its builtin ‘==’
> 
> I have a lot of respect for Ken Iverson, and a lot of admiration for language 
> designers willing to experiment with alternate paradigms.

This choice has significant(!!) costs: Fuzzy equality is not transitive:
One can get
a = b ∧ b = c ∧ a ≠ c
> 
> But keeping in mind that in APL, if you set ⎕ct to 0 you get an exact 
> comparison, can you find any quotes from Iverson saying that you should 
> *never* 
> perform exact equality comparisons?

There you go with your strawmen!
Remember it was you (and Chris) who expressed extreme positions:
“Pernicious myth” “FUD” etc
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Marko Rauhamaa]

Chris Angelico :

> On Thu, Jul 21, 2016 at 5:52 PM, Marko Rauhamaa  wrote:
>> A couple of related anecdotes involving integer errors.
>>
>> 1. I worked on a (video) product that had to execute a piece of code
>>every 7 µs or so. A key requirement was that the beat must not drift
>>far apart from the ideal over time. At first I thought the
>>traditional nanosecond resolution would be sufficient for the purpose
>>but then made a calculation:
>>
>> maximum rounding error = 0.5 ns/7 µs
>>= 70 µs/s
>>= 6 s/day
>>
>>That's why I decided to calculate the interval down to a femtosecond,
>>whose error was well within our tolerance.
>
> I'd be curious to know whether, had you used nanosecond resolution,
> you ever would have seen anything like that +/- 6s/day error. One
> convenient attribute of the real world [1] is that, unless there's a
> good reason for it to do otherwise [2], random error will tend to
> cancel out rather than accumulate. With error of +/- 0.5 ns, assume
> (for the sake of argument) that the actual error at each measurement
> is random.choice((-0.4, -0.3, -0.2, -0.1, 0.1, 0.2, 0.3, 0.4)) ns,

No, this is a systematic error due to the rounding of a rational number
to the nearest nanosecond interval. Systematic errors of this kind are a
real thing and not even all that uncommon.

> (I don't believe I've ever actually used a computer that's capable of
> nanosecond-accurate time calculations. Generally they return time in
> nanoseconds for consistency, but they won't return successive integer
> values. You must have been on some seriously high-end hardware -
> although that doesn't surprise me much, given that you were working on
> a video product.)

Nanosecond, even femtosecond, calculations are trivially simple integer
arithmetics that can be performed with pencil and paper. The hardware
was nothing fancy, and the timing error due to various hardware,
software and network realities was in the order of ±100 µs. That was
tolerable and could be handled through buffering. However, a cumulative
error of 6 seconds per day was *not* tolerable.

The calculations needed to be extremely simple because the interrupt
routine had to execute every 7 microseconds. You had to let the hardware
RTC do most of the work and do just a couple of integer operations in
the interrupt routine. In particular, you didn't have time to
recalculate and reprogram the RTC at every interrupt.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Chris Angelico]

On Thu, Jul 21, 2016 at 5:52 PM, Marko Rauhamaa  wrote:
> A couple of related anecdotes involving integer errors.
>
> 1. I worked on a (video) product that had to execute a piece of code
>every 7 µs or so. A key requirement was that the beat must not drift
>far apart from the ideal over time. At first I thought the
>traditional nanosecond resolution would be sufficient for the purpose
>but then made a calculation:
>
> maximum rounding error = 0.5 ns/7 µs
>= 70 µs/s
>= 6 s/day
>
>That's why I decided to calculate the interval down to a femtosecond,
>whose error was well within our tolerance.

I'd be curious to know whether, had you used nanosecond resolution,
you ever would have seen anything like that +/- 6s/day error. One
convenient attribute of the real world [1] is that, unless there's a
good reason for it to do otherwise [2], random error will tend to
cancel out rather than accumulate. With error of +/- 0.5 ns, assume
(for the sake of argument) that the actual error at each measurement
is random.choice((-0.4, -0.3, -0.2, -0.1, 0.1, 0.2, 0.3, 0.4)) ns,
with zero and the extremes omitted to make the calculations simpler.
In roughly twelve billion randomizations (86400 seconds divided by
7µs), the chances of having more than one billion more positive than
negative are... uhh actually, I don't know how to calculate
probabilities off numbers that big, but pretty tiny. So you're going
to have at least 5.5 billion negatives to offset your positives (or
positives to offset your negatives, same diff); more likely they'll be
even closer. So if you have (say) 5.5 to 6.5 ratio of signs, what
you're actually working with is half a second per day of accumulated
error - and I think you'd have a pretty tiny probability of even
*that* extreme a result. If it's more like 5.9 to 6.1, you'd have 0.1
seconds per day of error, at most. Plus, the same probabilistic
calculation can be done for days across a month, so even though the
theory would let you drift by three minutes a month, the chances of
shifting by even an entire second over that time are fairly slim.

This is something where I'd be more worried about systematic bias in
the code than anything from measurement or rounding error.

(I don't believe I've ever actually used a computer that's capable of
nanosecond-accurate time calculations. Generally they return time in
nanoseconds for consistency, but they won't return successive integer
values. You must have been on some seriously high-end hardware -
although that doesn't surprise me much, given that you were working on
a video product.)

ChrisA

[1] Believe you me, it has no shortage of INconvenient attributes, so
it's nice to have one swing the balance back a bit!
[2] If there's systematic error - if your 7 µs is actually averaging
7.25 µs - you need to deal with that separately.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Marko Rauhamaa]

Rustom Mody :
> The field of numerical analysis came into existence only because this
> fact multiplied by the fact that computers do their (inaccurate ≠
> inexact) computations billions of times faster than we do makes
> significance a very significant problem!

A couple of related anecdotes involving integer errors.

1. I worked on a (video) product that had to execute a piece of code
   every 7 µs or so. A key requirement was that the beat must not drift
   far apart from the ideal over time. At first I thought the
   traditional nanosecond resolution would be sufficient for the purpose
   but then made a calculation:

maximum rounding error = 0.5 ns/7 µs
   = 70 µs/s
   = 6 s/day

   That's why I decided to calculate the interval down to a femtosecond,
   whose error was well within our tolerance.

2. After running the LXDE GUI on my 32-bit Linux environment for some
   time, the CPU utilization monitor showed the CPU was mysteriously
   doing work 100% of the time. The only way out was to reboot the
   machine.

   After a few months and a few reboots, I investigated the situation
   more carefully. It turned out LXDE's CPU meter was reading jiffy
   counts from a textual /proc file with scanf("%ld"). Jiffies start
   from 0 at the time of the boot and increment every millisecond. Thus,
   the maximum signed 32-bit integer is reached in less than 25 days.

   When scanf("%ld") overflows, it sets the value to MAX_LONG. That
   effectively meant time stopped going forward and all rate
   calculations would shoot through the roof. This problem would not
   have occurred if the C standard consistently specified modulo
   arithmetics for integer operations.

   The solution was to use scanf("%lld") instead.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Steven D'Aprano]

On Thursday 21 July 2016 15:28, Rustom Mody wrote:

> On Wednesday, July 20, 2016 at 11:13:05 AM UTC+5:30, Steven D'Aprano wrote:
>> On Tuesday 19 July 2016 14:58, Rustom Mody wrote:
>> 
>> > So I again ask: You say «"Never compare floats for equality" is a
>> > pernicious myth»
>> 
>> It is the word *never* which makes it superstition. If people said "Take
>> care with using == for floats, its often not what you want" I would have no
>> argument with the statement.
>> 
>> I'd even (reluctantly) accept "usually not what you want". But "never" is
>> out- and-out cargo-cult programming.
> 
> You seem to not understand the realities of teaching.
> You (teacher in general) cannot say a saga; only epigrams

Is "Don't use exact equality unless you know what you're doing" enough of an 
epigram for you?


> You cannot transmit wisdom (even if you own some) just a bit of
> savviness/cleverness.

Maybe so, but that's no excuse for transmitting outright misinformation and 
superstition. In physics, dealing with motion in the presence of energy losses 
is hard, and beginning and intermediate levels of physics will generally 
explicitly or implicitly ignore friction and air resistance. But physics 
teachers do not teach that "air resistance doesn't exist; you mustn't try to 
take friction into account". They teach that it is a simplification.




> So let me ask the question again differently:
> How many errors happen by people not using ε-neighborhood checks instead of
> == checks How many errors happen by the opposite (mis)use?

Precisely 35, and 17812, respectively.


> IOW “myth”... ok “pernicious myth” Not
> 
> BTW APL whose main domain of application is scientific chooses to enshrine
> this —equality is ε-neighborhood checking not exact equality checking — into
> its builtin ‘==’

I have a lot of respect for Ken Iverson, and a lot of admiration for language 
designers willing to experiment with alternate paradigms.

But keeping in mind that in APL, if you set ⎕ct to 0 you get an exact 
comparison, can you find any quotes from Iverson saying that you should *never* 
perform exact equality comparisons?



> And
> 
http://www.dyalog.com/uploads/documents/Papers/tolerant_comparison/tolerant_comparison.htm

Nice resource, thank you.


> ε is spelt ⎕ct (Comparison Tolerance)
> And of course == is spelt =




-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Rustom Mody]

On Thursday, July 21, 2016 at 11:05:28 AM UTC+5:30, Chris Angelico wrote:
> On Thu, Jul 21, 2016 at 3:28 PM, Rustom Mody  wrote:
> > ε is spelt ⎕ct (Comparison Tolerance)
> > And of course == is spelt =
> 
> spelt is spelled spelled. Unless, of course, you mean the wheat variety.

Love it!
Though not everyone agrees (including Australians!)
http://english.stackexchange.com/questions/5712/spelt-vs-spelled

Anyway... Ive been collecting quines/self-referential statements for classes
on Theory of computation.

Like these from Douglas Hofstadter.
To which I’ll add yours

You cant have “your cake” and spell it “too”

You cant have your use and mention it too

If this sentence were in Chinese it would say something else

.siht ekil ti gnidaer eb d'uoy ,werbeH ni erew ecnetnes siht fI
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Rustom Mody]

On Wednesday, July 20, 2016 at 8:29:25 PM UTC+5:30, Marko Rauhamaa wrote:
> Chris Angelico :
> 
> > On Wed, Jul 20, 2016 at 11:54 PM, Marko Rauhamaa  wrote:
> >>  2. Floating-point numbers are *imperfect approximations* of real
> >> numbers. Even when real numbers are derived exactly,
> >> floating-point operations may introduce "lossy compression
> >> artifacts" that have to be compensated for in application
> >> programs.
> >
> > This is the kind of black FUD that has to be fought off. What
> > "compression artifacts" are introduced? The *only* lossiness in IEEE
> > binary floating-point arithmetic is rounding.
> 
> You are joining me in spreading the FUD. Yes, the immediate lossiness is
> rounding, but the effects of that rounding can result in atrocious
> accumulative errors in numeric calculations.
> 
> > Unless you are working with numbers that require more precision than
> > you have available, the result should be perfectly accurate.
> 
> Whoa, hold it there! Catastrophic cancellation ( https://en.wikipedia.org/wiki/Loss_of_significance>) is not a myth:

Whose lead para starts:


| Catastrophic cancellation… The effect is that the number of accurate 
| (significant) digits in the result is reduced unacceptably. Ways to avoid 
this 
| effect are studied in numerical analysis.

I would go a step further:
The field of numerical analysis came into existence only because this fact
multiplied by the fact that computers do their (inaccurate ≠ inexact) 
computations
billions of times faster than we do
makes significance a very significant problem!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Chris Angelico]

On Thu, Jul 21, 2016 at 3:28 PM, Rustom Mody  wrote:
> ε is spelt ⎕ct (Comparison Tolerance)
> And of course == is spelt =

spelt is spelled spelled. Unless, of course, you mean the wheat variety.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Rustom Mody]

On Wednesday, July 20, 2016 at 11:13:05 AM UTC+5:30, Steven D'Aprano wrote:
> On Tuesday 19 July 2016 14:58, Rustom Mody wrote:
> 
> > So I again ask: You say «"Never compare floats for equality" is a pernicious
> > myth»
> 
> It is the word *never* which makes it superstition. If people said "Take care 
> with using == for floats, its often not what you want" I would have no 
> argument 
> with the statement.
> 
> I'd even (reluctantly) accept "usually not what you want". But "never" is out-
> and-out cargo-cult programming.

You seem to not understand the realities of teaching.
You (teacher in general) cannot say a saga; only epigrams
You cannot transmit wisdom (even if you own some) just a bit of 
savviness/cleverness.

So let me ask the question again differently:
How many errors happen by people not using ε-neighborhood checks instead of == 
checks
How many errors happen by the opposite (mis)use?

IOW “myth”... ok “pernicious myth” Not

BTW APL whose main domain of application is scientific chooses to enshrine
this —equality is ε-neighborhood checking not exact equality checking — into 
its builtin ‘==’

And 
http://www.dyalog.com/uploads/documents/Papers/tolerant_comparison/tolerant_comparison.htm

ε is spelt ⎕ct (Comparison Tolerance)
And of course == is spelt =
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Marko Rauhamaa]

Chris Angelico :

> On Wed, Jul 20, 2016 at 11:54 PM, Marko Rauhamaa  wrote:
>>  2. Floating-point numbers are *imperfect approximations* of real
>> numbers. Even when real numbers are derived exactly,
>> floating-point operations may introduce "lossy compression
>> artifacts" that have to be compensated for in application
>> programs.
>
> This is the kind of black FUD that has to be fought off. What
> "compression artifacts" are introduced? The *only* lossiness in IEEE
> binary floating-point arithmetic is rounding.

You are joining me in spreading the FUD. Yes, the immediate lossiness is
rounding, but the effects of that rounding can result in atrocious
accumulative errors in numeric calculations.

> Unless you are working with numbers that require more precision than
> you have available, the result should be perfectly accurate.

Whoa, hold it there! Catastrophic cancellation (https://en.wikipedia.org/wiki/Loss_of_significance>) is not a myth:

   >>> 0.2 / (0.2 - 0.1)
   2.0
   >>> 0.2 / ((2e15 + 0.2) - (2e15 + 0.1))
   0.8

You can fall victim to the phenomenon when you collect statistics over a
long time. The cumulative sum of a measurement can grow very large,
which causes the naïve per-second rate calculation to become
increasingly bogus.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Chris Angelico]

On Wed, Jul 20, 2016 at 11:54 PM, Marko Rauhamaa  wrote:
>  2. Floating-point numbers are *imperfect approximations* of real
> numbers. Even when real numbers are derived exactly, floating-point
> operations may introduce "lossy compression artifacts" that have to
> be compensated for in application programs.

This is the kind of black FUD that has to be fought off. What
"compression artifacts" are introduced? The *only* lossiness in IEEE
binary floating-point arithmetic is rounding. (This is the bit where
someone like Steven is going to point out that there's something else
as well.) Unless you are working with numbers that require more
precision than you have available, the result should be perfectly
accurate. And there are other systems far less 'simple'. Can you
imagine this second assertion failing?

assert x <= y # if not, swap the values
assert x <= (x+y)/2 <= y

Because it can with decimal.Decimal, due to the way rounding happens in decimal.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Marko Rauhamaa]

Steven D'Aprano :

> I am not a good computer scientist. But Bruce Dawson *is* a good
> computer scientist:
>
> https://randomascii.wordpress.com/2014/01/27/theres-only-four-billion-f
> loatsso-test-them-all/
>
> Quote:
>
> Conventional wisdom says that you should never compare two floats
> for equality – you should always use an epsilon. Conventional
> wisdom is wrong.
>
> I’ve written in great detail about how to compare floating-point
> values using an epsilon, but there are times when it is just not
> appropriate. Sometimes there really is an answer that is correct,
> and in those cases anything less than perfection is just sloppy.
>
> So yes, I’m proudly comparing floats to see if they are equal.

The point of view in the linked article is very different from that of
most application programming that makes use of floating-point numbers.

Yes, if what you are testing or developing is a numeric or mathematical
package, you should test its numeric/mathematical soundness to the bit.
However, in virtually any other context, you have barely any use for
a floating-point equality comparison because:

 1. Floating-point numbers are an approximation of *real numbers*. Two
independently measured real numbers are never equal because under
any continuous probability distribution, the probability of any
given real number is zero. Only continuous ranges can have nonzero
probabilities.

 2. Floating-point numbers are *imperfect approximations* of real
numbers. Even when real numbers are derived exactly, floating-point
operations may introduce "lossy compression artifacts" that have to
be compensated for in application programs.

What you have to do exactly to compensate for these challenges depends
on the application, and is very easy to get wrong. However, if an
application programmer is using == to compare two floating-point data
values, it is almost certainly a mistake.

> Or you might be using a language like Javascript, which intentionally
> has only floats for numbers. That's okay, you can still perform exact
> integer arithmetic, so long as you stay within the bounds of ±2**16.
>
> Not even in Javascript do you need to write something like this:
>
> x = 0.0
> for i in range(20):
> x += 1.0
>
> assert abs(x - 20.0) <= 1e-16

Correct because Javascript makes an exactness guarantee of its integers
(I imagine).

In Python, I think it would usually be bad style to rely even on:

   1.0 + 1.0 == 2.0

It is very difficult to find a justification for that assumption in
Python's specifications. What we have:

   Floating-point numbers are represented in computer hardware as base 2
   (binary) fractions.
   https://docs.python.org/3/tutorial/floatingpoint.html>

   almost all platforms map Python floats to IEEE-754 “double precision”
   https://docs.python.org/3/tutorial/floatingpoint.html#represent
   ation-error>

   numbers.Real (float)
 These represent machine-level double precision floating point
 numbers. You are at the mercy of the underlying machine
 architecture (and C or Java implementation) for the accepted range
 and handling of overflow.
   https://docs.python.org/3/reference/datamodel.html#the-standar
   d-type-hierarchy>


I believe a Python implementation that would have:

   1.0 + 1.0 != 2.0

would not be in violation of Python's data model. In fact, even:

   1.0 != 1.0

might be totally conformant. For example, we could have a new underlying
real-number technology that stored the value in an *analogous* format
(say, an ultra-precise voltage level) and performed the calculations
using some fast, analogous circuitry.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Steven D'Aprano]

On Wed, 20 Jul 2016 05:09 pm, Antoon Pardon wrote:

> Op 20-07-16 om 07:42 schreef Steven D'Aprano:
>> Floating point maths is hard, thinking carefully about what you are doing
>> and whether it is appropriate to use == or a fuzzy almost-equal
>> comparison, or if equality is the right way at all.
>>
>> "But thinking is hard, can't you just tell me the answer?"
>>
>> No. But I can give some guidelines:
>>
>> Floating point arithmetic is deterministic, it doesn't just randomly mix
>> in error out of spite or malice. So in principle, you can always estimate
>> the rounding error from any calculation -- and sometimes there is none.
> 
> I would like to see a practical example of such an outcome.


[steve@ando ~]$ cd /home/steve/python/python-dev/3.4/Lib/test/
[steve@ando test]$ grep self.assertEqual test_statistics.py | wc -l
95


Not all of the 95 examples of using assertEqual apply to float values, but a
good proportion of them do. And if I were a better computer scientist,
there would probably be a lot more assertEquals in my code. A lot of the
time that I do a fuzzy comparison its because I'm too lazy or not good
enough to get a better result.

I am not a good computer scientist. But Bruce Dawson *is* a good computer
scientist:

https://randomascii.wordpress.com/2014/01/27/theres-only-four-billion-floatsso-test-them-all/

Quote:

Conventional wisdom says that you should never compare two floats
for equality – you should always use an epsilon. Conventional
wisdom is wrong.

I’ve written in great detail about how to compare floating-point
values using an epsilon, but there are times when it is just not
appropriate. Sometimes there really is an answer that is correct,
and in those cases anything less than perfection is just sloppy.

So yes, I’m proudly comparing floats to see if they are equal.



>> Arithmetic on integer-values (e.g. 1.0) is always exact, up to a limit of
>> either 2**53 or approximately 1e53, I forget which. (That's why most
>> Javascript programmers fail to notice that they don't have an integer
>> type.) So long as you're using operations that only produce integer
>> values from integer arguments (such as + - * // but not / ) then all
>> calculations are exact. It is a waste of time to do:
>>
>> x = 2.0
>> y = x*1002.0
>> is_equal(y, 2004.0, 1e-16)
>>
>> when you can just do y == 2004.0.
> 
> But why perforem integer arithmetics in floats, isn't that a waste of time
> too? I really see no reason to use floats if you know all your results
> will be integers.

In Python, it's probably neither harmful nor useful. The cost of dealing
with boxed values (objects rather than low-level machine values) will
probably outweigh any performance gain one way or the other.

But in lower-level languages, you might find that floating point arithmetic
is faster than integer arithmetic, if you can pass the work on to a FPU
instead of a (slow?) CPU. Or not. It depends on the machine.

Or you might be using a language like Javascript, which intentionally has
only floats for numbers. That's okay, you can still perform exact integer
arithmetic, so long as you stay within the bounds of ±2**16.

Not even in Javascript do you need to write something like this:

x = 0.0
for i in range(20):
x += 1.0

assert abs(x - 20.0) <= 1e-16




-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Marko Rauhamaa]

Antoon Pardon :
> But why perforem integer arithmetics in floats,

Conceptual and practical simplificity.

> isn't that a waste of time too?

Probably not, especially compared with the overhead of boxing.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Antoon Pardon]

Op 20-07-16 om 07:42 schreef Steven D'Aprano:
> Floating point maths is hard, thinking carefully about what you are doing and 
> whether it is appropriate to use == or a fuzzy almost-equal comparison, or if 
> equality is the right way at all.
>
> "But thinking is hard, can't you just tell me the answer?"
>
> No. But I can give some guidelines:
>
> Floating point arithmetic is deterministic, it doesn't just randomly mix in 
> error out of spite or malice. So in principle, you can always estimate the 
> rounding error from any calculation -- and sometimes there is none.

I would like to see a practical example of such an outcome.

> Arithmetic on integer-values (e.g. 1.0) is always exact, up to a limit of 
> either 2**53 or approximately 1e53, I forget which. (That's why most 
> Javascript 
> programmers fail to notice that they don't have an integer type.) So long as 
> you're using operations that only produce integer values from integer 
> arguments 
> (such as + - * // but not / ) then all calculations are exact. It is a waste 
> of 
> time to do:
>
> x = 2.0
> y = x*1002.0
> is_equal(y, 2004.0, 1e-16)
>
> when you can just do y == 2004.0.

But why perforem integer arithmetics in floats, isn't that a waste of time too?
I really see no reason to use floats if you know all your results will be 
integers.

-- 
Antoon.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Chris Angelico]

On Wed, Jul 20, 2016 at 3:42 PM, Steven D'Aprano
 wrote:
> Arithmetic on integer-values (e.g. 1.0) is always exact, up to a limit of
> either 2**53 or approximately 1e53, I forget which. (That's why most 
> Javascript
> programmers fail to notice that they don't have an integer type.)

It's 2**53, because 64-bit floats use a 53-bit mantissa (52-bits
stored and an implicit 1 at the beginning, although I can never
remember how denormals are represented). Works out to a bit under
1e16. AIUI asm.js offers a 32-bit integer type, which in fall-back
mode is represented with the native "Number" type; for values that
could be stored in a 32-bit integer, a 64-bit float is perfectly
accurate (just stupidly inefficient compared to a real integer type).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list