@listserver.u2ug.org
Sent: Monday, October 10, 2011 6:35 PM
Subject: Re: [U2] The math just doesn't work.
I thought that large integer addition is why the functions SADD, SCMP and
SDIV were invented?
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
- Original Message -
*From:* bre...@gpmdev.co.uk
*To:* u2-users@listserver.u2ug.org u2-users@listserver.u2ug.org
*Date:* 10/10/2011 1:16 AM
*Subject:* Re: [U2] The math just doesn't work.
I also thought: great explanation and nodded sagely when I read this.
However
I also thought: great explanation and nodded sagely when I read this.
However, consider this code:
NUL = ''
CM = ','
A = -409071.8775: CM: 475000: CM: -652413: CM: 652413: CM: -475000: CM:
409071.8775
*
*
TEST.VMC = 1
TEST.TOTAL = 0
*
LOOP UNTIL
Coldstream Lane, Hardingstone, Northampton NN4 6DB, England
+44 (0)1604-709200
- Original Message -
From: Brett Callacher bre...@gpmdev.co.uk
To: u2-users@listserver.u2ug.org
Sent: Monday, October 10, 2011 9:16 AM
Subject: Re: [U2] The math just doesn't work.
I also thought: great
-
From: Brett Callacher bre...@gpmdev.co.uk
To: u2-users@listserver.u2ug.org
Sent: Monday, October 10, 2011 9:16 AM
Subject: Re: [U2] The math just doesn't work.
I also thought: great explanation and nodded sagely when I read this.
However, consider this code:
NUL
Hi Brett,
The realisation for me here is that PRECISION has nothing to do
with the precision of the calculation, it only effects display.
Correct. All arithetic operations work at the precision of the underlying
hardware. PRECISION applies when converting a floating point number to a
List
Subject: Re: [U2] The math just doesn't work.
Hi Brett,
The realisation for me here is that PRECISION has nothing to do
with the precision of the calculation, it only effects display.
Correct. All arithetic operations work at the precision of the underlying
hardware. PRECISION applies when
On 10/10/2011 8:12 AM, Israel, John R. wrote:
I am coming late to this party, but wanted to throw out a bug that was just
reported to Rocket. We found that PERCISION was causing a calculation to
round in some cases and truncate in others. I am not sure where this stands
because someone
Egerton
Sent: Monday, October 10, 2011 8:31 AM
To: U2 Users List
Subject: Re: [U2] The math just doesn't work.
On 10/10/2011 8:12 AM, Israel, John R. wrote:
I am coming late to this party, but wanted to throw out a bug that was just
reported to Rocket. We found that PERCISION was causing
On 10/10/11 13:31, Allen Egerton wrote:
On 10/10/2011 8:12 AM, Israel, John R. wrote:
I am coming late to this party, but wanted to throw out a bug that was just
reported to Rocket. We found that PERCISION was causing a calculation to
round in some cases and truncate in others. I am not
On 10/10/2011 12:31 PM, Wols Lists wrote:
On 10/10/11 13:31, Allen Egerton wrote:
On 10/10/2011 8:12 AM, Israel, John R. wrote:
I am coming late to this party, but wanted to throw out a bug that was just
reported to Rocket. We found that PERCISION was causing a calculation to
round in some
- Original Message -
*From:* bre...@gpmdev.co.uk
*To:* u2-users@listserver.u2ug.org u2-users@listserver.u2ug.org
*Date:* 10/10/2011 1:16 AM
*Subject:* Re: [U2] The math just doesn't work.
I also thought: great explanation and nodded sagely when I read this.
However
Hi Mecki,
I actually wonder if this feature is flavour specific?
Pick didn't use floating point arithmetic and IIRC the default
precision used to be 4.
PRECISION affects the number of decimal places in the result when converting
a floating point value to a character string. All internal
ICONV everything MD2 and check that for equality
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of George
Hammerle
Sent: 29 September 2011 13:30
To: u2-users@listserver.u2ug.org
Subject: [U2] The math just doesn't
Hi George,
As a general rule in programming, comparison of floating point values for
equality should be avoided. This is because, just as we cannot write the
number one third accurately in decimal notation, so the IEEE floating point
format used by computer systems cannot store numbers
= 4203.09, C= 4203.09
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Martin Phillips
Sent: 29 September 2011 13:46
To: U2 Users List
Subject: Re: [U2] The math just doesn't work.
Hi George,
As a general rule
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Brian Leach
Sent: Thursday, September 29, 2011 6:21 AM
To: 'U2 Users List'
Subject: Re: [U2] The math just doesn't work.
And just to add to Martin's excellent answer - the normal workround for
any
computer system is to use integral values
();
}
}
}
Result -
You lie, A+B = 4203.09, C= 4203.09
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Martin
Phillips
Sent: 29 September 2011 13:46
To: U2 Users List
Subject: Re: [U2] The math just doesn't work
- Original Message -
*From:* harold.o...@clark.wa.gov
*To:* U2 Users List u2-users@listserver.u2ug.org
*Date:* 9/29/2011 8:40 AM
*Subject:* Re: [U2] The math just doesn't work.
Running Martin's code on a Universe system, got:
(A+B) = C? YOU
-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Oaks, Harold
Sent: Thursday, September 29, 2011 8:40 AM
To: U2 Users List
Subject: Re: [U2] The math just doesn't work.
Running Martin's code on a Universe system, got:
(A+B) = C? YOU ROCK
A = 4096.9, B
to it in the Administering Universe
guide.
Harold
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Mecki
Foerthmann
Sent: Thursday, September 29, 2011 8:53 AM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] The math just doesn't work
: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Bill Haskett
Sent: Thursday, September 29, 2011 9:02 AM
To: U2 Users List
Subject: Re: [U2] The math just doesn't work.
I tried this on UD v7.2.7 on Windows 2008 R2. I added the following code
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Mecki
Foerthmann
Sent: Thursday, September 29, 2011 8:53 AM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] The math just doesn't work.
You didn't read Martin's answer
23 matches
Mail list logo