Re: Phobos unit testing uncovers a CPU bug
Andrew Wiley Wrote: What programmer? What algorithm? As far as I can tell, this was found when testing a library explicitly for accuracy, not in an application, so your argument doesn't apply. Hmm... Really... I've messed this.
Re: Phobos unit testing uncovers a CPU bug
Kagamin wrote: It's not me, it's the programmer. He was disgusted that his algorithm produced garbage, which means, the error was unacceptable. Mat be it was 1%, may be 80%, I don't, that was his decision, that the result was unacceptable. The bug description assumes the problem was in the last bit, which means, he wanted precision higher than the machine precision. This thread is about a bug in a CPU's floating point implementation. Regardless of which bit of the floating point representation is affected, it's a bug. Ali
Re: Phobos unit testing uncovers a CPU bug
Walter Bright Wrote: How do you decide how many bits should be enough for any algorithm? The thing is, the FPU has 53 bits of precision and so ought to be correct to the last bit. It's not me, it's the programmer. He was disgusted that his algorithm produced garbage, which means, the error was unacceptable. Mat be it was 1%, may be 80%, I don't, that was his decision, that the result was unacceptable. The bug description assumes the problem was in the last bit, which means, he wanted precision higher than the machine precision.
Re: Phobos unit testing uncovers a CPU bug
On Tue, Nov 30, 2010 at 1:43 PM, Kagamin s...@here.lot wrote: Walter Bright Wrote: How do you decide how many bits should be enough for any algorithm? The thing is, the FPU has 53 bits of precision and so ought to be correct to the last bit. It's not me, it's the programmer. He was disgusted that his algorithm produced garbage, which means, the error was unacceptable. Mat be it was 1%, may be 80%, I don't, that was his decision, that the result was unacceptable. The bug description assumes the problem was in the last bit, which means, he wanted precision higher than the machine precision. What programmer? What algorithm? As far as I can tell, this was found when testing a library explicitly for accuracy, not in an application, so your argument doesn't apply.
Re: Phobos unit testing uncovers a CPU bug
import std.conv: to;
void main() {
auto r = to!real(0x1.0076fc5cc7933866p+40L);
auto d = to!double(0x1.0076fc5cc7933866p+40L);
auto f = to!float(0x1.0076fc5cc7933866p+40L);
}
Regarding unit tests, I should really use them :(
Yep, and DbC too, and compile your D code with -w.
Bye,
bearophile
I have an unrelated question, this is not a criticism but an honest one.
Why don't you write these 3 lines like:
auto r = to!real (0x1.0076fc5cc7933866p+40L);
auto d = to!double(0x1.0076fc5cc7933866p+40L);
auto f = to!float (0x1.0076fc5cc7933866p+40L);
Thank you.
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Phobos unit testing uncovers a CPU bug
The same question goes to you. Why do you call this bug? It is approximate, but approximation is not an undefined behavior. It is same as 2 + 1 = 4. -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Phobos unit testing uncovers a CPU bug
On 27/11/2010 06:26, Don wrote: I haven't seen any examples of values which are calculated differently between the processors. I only found one vague reference in a paper from CERN. And because of that comment, I've once again checked http://hasthelargehadroncolliderdestroyedtheworldyet.com/ , just to make sure... :P CERN better be aware of that stuff! :D -- Bruno Medeiros - Software Engineer
Re: Phobos unit testing uncovers a CPU bug
so: I have an unrelated question, this is not a criticism but an honest one. Why don't you write these 3 lines like: auto r = to!real (0x1.0076fc5cc7933866p+40L); auto d = to!double(0x1.0076fc5cc7933866p+40L); auto f = to!float (0x1.0076fc5cc7933866p+40L); I have a certain stylistic rule regarding how function calls are written in my code, and that rule is more important than such alignment. Adding spaces where I don't expect them to be slows down my visual parsing of the code. Generally in my code I've found that such kind of alignment is a waste of my time, if I change something I may need to waste time realigning things. Bye, bearophile
Re: Phobos unit testing uncovers a CPU bug
On Monday, November 29, 2010 12:36:00 bearophile wrote: so: I have an unrelated question, this is not a criticism but an honest one. Why don't you write these 3 lines like: auto r = to!real (0x1.0076fc5cc7933866p+40L); auto d = to!double(0x1.0076fc5cc7933866p+40L); auto f = to!float (0x1.0076fc5cc7933866p+40L); I have a certain stylistic rule regarding how function calls are written in my code, and that rule is more important than such alignment. Adding spaces where I don't expect them to be slows down my visual parsing of the code. Generally in my code I've found that such kind of alignment is a waste of my time, if I change something I may need to waste time realigning things. LOL. I couldn't figure out what was different about those lines (the extra space apparently; the fact that the text isn't monospaced makes it harder to see though). But no one's over going to get people to agree on spacing any more than they'll get them to agree on braces. Personally, I generally don't have extraneous spaces but will periodically line up variable declarations with regards to names (which would require no work here because they're all auto and line up anyway). Other folks I know insist on always putting extra spaces after keywords like if and/or putting extra spaces before or after parens. Trying to convince anyone about spacing is a waste of time, and I'd say that suggesting that spacing be altered in code is generally a waste of time unless there's something abnormally bad about it (like 10 spaces after every paren or something ridiculous like that). - Jonathan M Davis
Re: Phobos unit testing uncovers a CPU bug
Walter Bright Wrote: Really? I think, the answer is neither correct nor wrong. It's approximate. The rules for rounding the mathematical value to the representation are precise, and so there is such a thing as the correctly rounded result and the wrong result. Well, maybe, but the result fits well in machine precision (which is equal to the last bit). And if this precision is not enough for the algorithm, then the algorithm is incorrect on any x87 FPU, and its output is garbage.
Re: Phobos unit testing uncovers a CPU bug
On Monday 29 November 2010 21:30:31 s...@so.do wrote: LOL. I couldn't figure out what was different about those lines (the extra space apparently; the fact that the text isn't monospaced makes it harder to see though). But no one's over going to get people to agree on spacing any more than they'll get them to agree on braces. Personally, I generally don't have extraneous spaces but will periodically line up variable declarations with regards to names (which would require no work here because they're all auto and line up anyway). Other folks I know insist on always putting extra spaces after keywords like if and/or putting extra spaces before or after parens. Trying to convince anyone about spacing is a waste of time, and I'd say that suggesting that spacing be altered in code is generally a waste of time unless there's something abnormally bad about it (like 10 spaces after every paren or something ridiculous like that). - Jonathan M Davis IMHO this is a good example why inconsistencies are sometimes necessary. I don't put spaces into random places either but this example is crying for attention :) If you say so. I don't see any problem with it. I can see why you'd want to add the extra spaces (at least, once I put it in an editor with a monospaced font), but I wouldn't have thought that it would be worth calling someone on it. It just seems nitpicky, honestly. - Jonathan M Davis
Re: Phobos unit testing uncovers a CPU bug
but I wouldn't have thought that it would be worth calling someone on it. It just seems nitpicky, honestly. - Jonathan M Davis I am sorry if it seems that way, wasn't my intention. -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Phobos unit testing uncovers a CPU bug
Don Wrote: The Intel CPU gives the correct answer, but AMD's is wrong. They should both give the correct result. Really? I think, the answer is neither correct nor wrong. It's approximate. If your friend's program operates on ~0x1p+40 values and critically depends on on the value of the last bit, then double precision doesn't suit his needs (on both Intel and AMD), he should take a couple of classes on computational theory and rewrite his algorithm, or use arithmetic with higher precision.
Re: Phobos unit testing uncovers a CPU bug
If it happens once its a bug, if its repeatable its a feature ;-) -=mike=- Kagamin s...@here.lot wrote in message news:icth6h$1nq...@digitalmars.com... Don Wrote: The Intel CPU gives the correct answer, but AMD's is wrong. They should both give the correct result. Really? I think, the answer is neither correct nor wrong. It's approximate. If your friend's program operates on ~0x1p+40 values and critically depends on on the value of the last bit, then double precision doesn't suit his needs (on both Intel and AMD), he should take a couple of classes on computational theory and rewrite his algorithm, or use arithmetic with higher precision.
Re: Phobos unit testing uncovers a CPU bug
Kagamin wrote: Don Wrote: The Intel CPU gives the correct answer, but AMD's is wrong. They should both give the correct result. Really? I think, the answer is neither correct nor wrong. It's approximate. The rules for rounding the mathematical value to the representation are precise, and so there is such a thing as the correctly rounded result and the wrong result. An FPU should strive to always produce the correctly rounded result.
Re: Phobos unit testing uncovers a CPU bug
On Nov 27, 10 05:25, Simen kjaeraas wrote: Don nos...@nospam.com wrote: The difference was discovered through the unit tests for the mathematical Special Functions which will be included in the next compiler release. Discovery of the discrepancy happened only because of several features of D: - built-in unit tests (encourages tests to be run on many machines) - built-in code coverage (the tests include extreme cases, simply because I was trying to increase the code coverage to high values) - D supports the hex format for floats. Without this feature, the discrepancy would have been blamed on differences in the floating-point conversion functions in the C standard library. This experience reinforces my belief that D is an excellent language for scientific computing. This sounds like a great sales argument. Gives us some bragging rights. :p Thanks to David Simcha and Dmitry Olshansky for help in tracking this down. Great job! Now, which of the results is correct, and has AMD and Intel been informed? Intel is correct. yl2x(0x1.0076fc5cc7933866p+40L, LN2) == log(9240117798188457011/8388608) == 0x1.bba4a9f774f49d0a64ac5666c969fd8ca8e...p+4 ^
Re: Phobos unit testing uncovers a CPU bug
On 26.11.2010 23:02, Don wrote:
The code below compiles to a single machine instruction, yet the
results are CPU manufacturer-dependent.
import std.math;
void main()
{
assert( yl2x(0x1.0076fc5cc7933866p+40L, LN2)
== 0x1.bba4a9f774f49d0ap+4L); // Pass on Intel, fails on AMD
}
The results for yl2x(0x1.0076fc5cc7933866p+40L, LN2) are:
Intel: 0x1.bba4a9f774f49d0ap+4L
AMD:0x1.bba4a9f774f49d0cp+4L
The least significant bit is different. This corresponds only to a
fraction of a bit (that is, it's hardly important for accuracy. For
comparison, sin and cos on x86 lose nearly sixty bits of accuracy in
some cases!). Its importance is only that it is an undocumented
difference between manufacturers.
The difference was discovered through the unit tests for the
mathematical Special Functions which will be included in the next
compiler release. Discovery of the discrepancy happened only because
of several features of D:
- built-in unit tests (encourages tests to be run on many machines)
- built-in code coverage (the tests include extreme cases, simply
because I was trying to increase the code coverage to high values)
- D supports the hex format for floats. Without this feature, the
discrepancy would have been blamed on differences in the
floating-point conversion functions in the C standard library.
This experience reinforces my belief that D is an excellent language
for scientific computing.
Thanks to David Simcha and Dmitry Olshansky for help in tracking this
down.
Glad to help!
I was genuinely intrigued because not more then a few weeks ago I
discussed with a friend of mine a possibility of differences in FP
calculations of AMD vs Intel.
You see, his scientific app yielded different results when working at
home/at work, which is a frustrating experience. Since that's exactly
same binary, written in Delphi (no C run-time involved and so on) and
environment is pretty much the same... I suggested to check CPU vendors
just in case... of course, different.
In the meantime, I sort of ported the test case to M$ c++ inline asm
and posted it on AMD forums, let's see what they have to say.
http://forums.amd.com/forum/messageview.cfm?catid=319threadid=142893enterthread=y
http://forums.amd.com/forum/messageview.cfm?catid=319threadid=142893enterthread=y
--
Dmitry Olshansky
Re: Phobos unit testing uncovers a CPU bug
Don Wrote: The great tragedy was that an early AMD processor gave much accurate sin and cos than the 387. But, people complained that it was different from Intel! So, their next processor duplicated Intel's hopelessly wrong trig functions. The same question goes to you. Why do you call this bug?
Re: Phobos unit testing uncovers a CPU bug
On 28-11-2010 5:49, Dmitry Olshansky wrote:
On 26.11.2010 23:02, Don wrote:
The code below compiles to a single machine instruction, yet the
results are CPU manufacturer-dependent.
import std.math;
void main()
{
assert( yl2x(0x1.0076fc5cc7933866p+40L, LN2)
== 0x1.bba4a9f774f49d0ap+4L); // Pass on Intel, fails on AMD
}
The results for yl2x(0x1.0076fc5cc7933866p+40L, LN2) are:
Intel: 0x1.bba4a9f774f49d0ap+4L
AMD: 0x1.bba4a9f774f49d0cp+4L
The least significant bit is different. This corresponds only to a
fraction of a bit (that is, it's hardly important for accuracy. For
comparison, sin and cos on x86 lose nearly sixty bits of accuracy in
some cases!). Its importance is only that it is an undocumented
difference between manufacturers.
The difference was discovered through the unit tests for the
mathematical Special Functions which will be included in the next
compiler release. Discovery of the discrepancy happened only because
of several features of D:
- built-in unit tests (encourages tests to be run on many machines)
- built-in code coverage (the tests include extreme cases, simply
because I was trying to increase the code coverage to high values)
- D supports the hex format for floats. Without this feature, the
discrepancy would have been blamed on differences in the
floating-point conversion functions in the C standard library.
This experience reinforces my belief that D is an excellent language
for scientific computing.
Thanks to David Simcha and Dmitry Olshansky for help in tracking this
down.
Glad to help!
I was genuinely intrigued because not more then a few weeks ago I
discussed with a friend of mine a possibility of differences in FP
calculations of AMD vs Intel.
You see, his scientific app yielded different results when working at
home/at work, which is a frustrating experience. Since that's exactly
same binary, written in Delphi (no C run-time involved and so on) and
environment is pretty much the same... I suggested to check CPU vendors
just in case... of course, different.
In the meantime, I sort of ported the test case to M$ c++ inline asm
and posted it on AMD forums, let's see what they have to say.
http://forums.amd.com/forum/messageview.cfm?catid=319threadid=142893enterthread=y
http://forums.amd.com/forum/messageview.cfm?catid=319threadid=142893enterthread=y
http://forums.amd.com/forum/messageview.cfm?catid=29threadid=135771
This post also talks about a fyl2x bug. Wonder if it's the same bug.
L.
Re: Phobos unit testing uncovers a CPU bug
Kagamin wrote: Don Wrote: The great tragedy was that an early AMD processor gave much accurate sin and cos than the 387. But, people complained that it was different from Intel! So, their next processor duplicated Intel's hopelessly wrong trig functions. The same question goes to you. Why do you call this bug? The Intel CPU gives the correct answer, but AMD's is wrong. They should both give the correct result.
Re: Phobos unit testing uncovers a CPU bug
== Quote from Don (nos...@nospam.com)'s article
The code below compiles to a single machine instruction, yet the results
are CPU manufacturer-dependent.
import std.math;
void main()
{
assert( yl2x(0x1.0076fc5cc7933866p+40L, LN2)
== 0x1.bba4a9f774f49d0ap+4L); // Pass on Intel, fails on AMD
}
The results for yl2x(0x1.0076fc5cc7933866p+40L, LN2) are:
Intel: 0x1.bba4a9f774f49d0ap+4L
AMD:0x1.bba4a9f774f49d0cp+4L
The least significant bit is different. This corresponds only to a
fraction of a bit (that is, it's hardly important for accuracy. For
comparison, sin and cos on x86 lose nearly sixty bits of accuracy in
some cases!). Its importance is only that it is an undocumented
difference between manufacturers.
The difference was discovered through the unit tests for the
mathematical Special Functions which will be included in the next
compiler release. Discovery of the discrepancy happened only because of
several features of D:
- built-in unit tests (encourages tests to be run on many machines)
- built-in code coverage (the tests include extreme cases, simply
because I was trying to increase the code coverage to high values)
- D supports the hex format for floats. Without this feature, the
discrepancy would have been blamed on differences in the floating-point
conversion functions in the C standard library.
This experience reinforces my belief that D is an excellent language for
scientific computing.
Thanks to David Simcha and Dmitry Olshansky for help in tracking this down.
Must have made you smile ;)
Slightly related, do you have some code to convert a hex float string to float?
I think the hex format is a nice compromise between size and readability.
Regarding unit tests, I should really use them :(
I use std2 in my D1 project and a few of std2's unit tests fail, so I run my
tests() manually..
Re: Phobos unit testing uncovers a CPU bug
Don wrote: The code below compiles to a single machine instruction, yet the results are CPU manufacturer-dependent. This is awesome work, Don. Kudos to you, David and Dmitry. BTW, I've read that fine-grained CPU detection can be done, beyond what CPUID gives, by examining slight differences in FPU results. I expect that *, +, -, / should all give exactly the same answers. But the transcendentals, and obviously yl2x, vary.
Re: Phobos unit testing uncovers a CPU bug
%u wrote: Slightly related, do you have some code to convert a hex float string to float? Hex float literals are supported by D.
Re: Phobos unit testing uncovers a CPU bug
Walter: %u wrote: Slightly related, do you have some code to convert a hex float string to float? Hex float literals are supported by D. hex float string != Hex float literal. Bye, bearophile
Re: Phobos unit testing uncovers a CPU bug
Walter Bright wrote: Don wrote: The code below compiles to a single machine instruction, yet the results are CPU manufacturer-dependent. This is awesome work, Don. Kudos to you, David and Dmitry. BTW, I've read that fine-grained CPU detection can be done, beyond what CPUID gives, by examining slight differences in FPU results. I expect that *, +, -, / should all give exactly the same answers. But the transcendentals, and obviously yl2x, vary. I believe that would have once been possible, I doubt it's true any more. Basic arithmetic and sqrt all give correctly rounded results, so they're identical on all processors. The 387 gives greatly improved accuracy, compared to the 287. But AFAIK there have not been intentional changes since then. The great tragedy was that an early AMD processor gave much accurate sin and cos than the 387. But, people complained that it was different from Intel! So, their next processor duplicated Intel's hopelessly wrong trig functions. I haven't seen any examples of values which are calculated differently between the processors. I only found one vague reference in a paper from CERN.
