We're on 6.3 Patch 18 I've been doing some testing since last night and it seems like it's working fine until it multiplies by 2080.08 If I just set a field with (((19.2300 * .040000) * 1.000001) + 19.2300) The end result is correct. Once it's multiplies by 2080.08 it is wrong. Robert, I tried hard coding in the number 1.000001 and it still came out wrong :(
On Wed, 17 Jan 2007 09:36:24 -0600, Lucero, Michelle - IST contractor <[EMAIL PROTECTED]> wrote: >Hi, Dylan: > >This issue sounds somewhat familiar and believe it may have been >reported as a bug. What version of Mid-Tier, ARS System and User Tool >are you on? > >I ask because there are several calculation type bugs reported in 6.3. >When I put in "calculation" as a keyword in the support site's Product >Defects search, it returned 30 entries. > >Mid-Tier 7.0 also carries its own set of mathematical issues. On >another note, I was told that the modulo operator issue is suppose to be >fixed in a 7.0.1 Mid-Tier patch > >Michelle > >________________________________ > >From: Action Request System discussion list(ARSList) >[mailto:[EMAIL PROTECTED] On Behalf Of Babineau, Robert (xwave) >Sent: Wednesday, January 17, 2007 7:37 AM >To: [email protected] >Subject: Re: Mid Tier vs User Tool Calculation difference? > > >** >I know this is an incorrect calculation, but have you tried changing >your 1.000000 to a 1.0000001 or even taking it out completely? Remedy >may be trying to CAST that number into an Integer instead of a Real >along the way. > >(((19.2300 * .040000) * 1.000001) + 19.2300) * 2080.08 > >Robert Babineau, P.Eng. >xwave >[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> >Come visit us at www.xwave.com <http://www.xwave.com/> > > >________________________________ > >From: Wheeler, Dylan [mailto:[EMAIL PROTECTED] >Sent: Wednesday, January 17, 2007 12:40 AM >Subject: Mid Tier vs User Tool Calculation difference? > > >** > >Hi Listers, >I'm having a problem with a calculation I'm doing. I'm using an AL set >field to do a calculation and set a real number field that is set to a >precision of 2. The user tool calculates the right value but it looks >like the mid tier rounds the value to the nearest whole number. The >calculation is like this > >(((19.2300 * .040000) * 1.000000) + 19.2300) * 2080.08 >User tool comes out with 41599.94 and mid tier comes out with 41600.00 > >Anyone run into this before or know what to do to correct it? > >------------- > >Dylan Wheeler >Production Control Analyst Principle >IT Operations >Downey Savings & Loan Association, F.A. >Email: [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> > >This message and any attachments are for the intended recipient(s) only >and may contain privileged, confidential and/or proprietary information >about Downey Savings or its customers, which Downey Savings does not >intend to disclose to the public. If you received this message by >mistake, please notify the sender by reply e-mail and delete the message >and attachments. > > > >This message and any attachments are for the intended recipient(s) only >and may contain privileged, confidential and/or proprietary information >about Downey Savings or its customers, which Downey Savings does not >intend to disclose to the public. If you received this message by >mistake, please notify the sender by reply e-mail and delete the message >and attachments. > >__20060125_______________________This posting was submitted with HTML in >it___ __20060125_______________________This posting was submitted with >HTML in it___ > >___________________________________________________________________________ ____ >UNSUBSCRIBE or access ARSlist Archives at www.arslist.org ARSlist:"Where the Answers Are" > _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org ARSlist:"Where the Answers Are"
