Thanks Doug, Misi, Ben. I thought it traditional mathematical/arithmetic meaning of real or decimal numbers where what you enter would be what it is saved as.
Joe -----Original Message----- From: Action Request System discussion list(ARSList) [mailto:[email protected]] On Behalf Of Mueller, Doug Sent: Wednesday, January 23, 2013 3:13 PM To: [email protected] Subject: Re: Real number fields and the Precision property.. Just to make sure there is no confusion... One sentence in the message below "Decimal fields can not stores the exact..." is incorrect. It should be Decimal fields CAN store the exact digits you enter. This is just a typo as the paragraph is correct, but I wanted to make sure that there was no confusion. Real numbers are "floating point" numbers. They are stored in a "floating point" format that consists of the number and a "power of ten to say where the decimal point goes". So, 1 trillion is stored as 1 and to the 12th power of 10. Precision is the number of digits of accuracy of the initial number -- regardless of where the decimal point falls. In addition, "floating point" numbers are not guaranteed to be exact. Their storage format is an approximate format. If you have a number that has 8 or more digits, you will frequently find that the stored number is just slightly different in the final digits than you saved. This is typical and expected of this data format. Decimal numbers are really character strings to make sure that they are EXACT. One trillion would be stored as 1000000000000. Decimal numbers require more overhead and they have a limit of about 30 digits total so you cannot represent say 1.23 x 10 to the 500 like you can with floating point numbers. Precision is a display property only for real fields. Floating point storage is a fixed format and there is no precision within the format. Decimal precision on the other hand is enforced and modifies the value stored to be consistent with the defined precision. The values are stored as strings and can therefore be adjusted to the exact definition specified. Also note.... Currency fields use the Decimal type format because any currency calculations must be exact. This is just in case you were concerned with the mention of "exact" and "not exact" of the real and decimal data types. Doug Mueller -----Original Message----- From: Action Request System discussion list(ARSList) [mailto:[email protected]] On Behalf Of Misi Mladoniczky Sent: Wednesday, January 23, 2013 12:36 AM To: [email protected] Subject: Re: Real number fields and the Precision property.. Hi, A real number precision is just a display property. The would be stored as a floating point number in the database. This means that the number might or might not be able to store the exact value you intend it to store. Decimal fields can not stores the exact digits you enter, and you can not store any other precision than the one you specified for the field. It is stored as a string in the database. I use decimal fields a lot, as they are exact. When using Real values, you have to use the ROUND-functions much more. In you rexample. Try to store 0.1234567891. I would be that it will store more decimal digits as you have fewer significant digits in total. Best Regards - Misi, RRR AB, http://rrr.se > I've barely (if at all) used real or decimal fields in a long time, so I do > not recall if this was ever working like how I would expect it to, bearing > in mind the true mathematical meaning of the term precision used with > decimal or real numbers. > > > > How does the AR System store Real Number fields when a certain precision > level is selected as its property? For e.g. if I choose a Real Number field > to have a precision of 10, if I stored a number 2341.1234567891, I would > expect Remedy to display 2341.1234567891 as the value that was stored after > the commit. However it displays 2341.1234570000 after rounding up, which I > find to be strange. And in the database, it stores only 2341.123457. > > > > Precision, thus I have noticed is the total count of digits BOTH before and > after the decimal point in Remedy AND NOT the count of digits after the > decimal point which according to me would be a traditional definition. Is > this a bug? > > > > I am messing around with AR System 7.6.04 Patch 003 installed on Windows > 2008 and using Ms-SQL 2008 R2 as the underlying DB for those who want to > know my environment. > > > > Cheers > > > > Joe _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org "Where the Answers Are, and have been for 20 years"
