Re: [HACKERS] Purpose of rscale/dscale in NUMERIC?
Tom Lane wrote: What is the reason for maintaining separate rscale and dscale values in numeric variables? I am finding that this arrangement leads to some odd results, for example this: regression=# select (exp(ln(2.0)) - 2.0); ?column? - -0. (1 row) regression=# select (exp(ln(2.0)) - 2.0) * 10; ?column? - -0.0010 (1 row) The difference between rscale and dscale allows some hidden digits to be carried along in an expression result, and then possibly exposed later. This seems pretty nonintuitive for an allegedly exact calculational datatype. ISTM the policy should be what you see is what you get - no hidden digits. That would mean there's no need for separating rscale and dscale, so I'm wondering why they were put in to begin with. You need to carry around a decent number of digits when you divide already. Exposing them in a manner that numericcol(15,2) / 3.0 all of the sudden displays 16 or more digits isn't much more intuitive. But carrying around only 2 here leads to nonintuitively fuzzy results on the other hand. It only applies to division and higher functions, and these are not exact if you calculate the result and represent it decimal. They never have been. So to answer your question, they are there to make the NUMERIC datatype useful for non-exact stuff too. You can expect an exact result where an exact representation in decimal can be expected. Where this is not the case, you get a good approximation. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Purpose of rscale/dscale in NUMERIC?
Jan Wieck [EMAIL PROTECTED] writes: Tom Lane wrote: What is the reason for maintaining separate rscale and dscale values in numeric variables? You need to carry around a decent number of digits when you divide already. Exposing them in a manner that numericcol(15,2) / 3.0 all of the sudden displays 16 or more digits isn't much more intuitive. But carrying around only 2 here leads to nonintuitively fuzzy results on the other hand. Certainly you need extra guard digits while you do the calculation. What I'm wondering is why the delivered result would have hidden digits in it. If they're accurate, why not show them? If they're not accurate (which they're not, at least in the case I showed) why is it a good idea to let them escape? regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Purpose of rscale/dscale in NUMERIC?
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: Tom Lane wrote: What is the reason for maintaining separate rscale and dscale values in numeric variables? You need to carry around a decent number of digits when you divide already. Exposing them in a manner that numericcol(15,2) / 3.0 all of the sudden displays 16 or more digits isn't much more intuitive. But carrying around only 2 here leads to nonintuitively fuzzy results on the other hand. Certainly you need extra guard digits while you do the calculation. What I'm wondering is why the delivered result would have hidden digits in it. If they're accurate, why not show them? If they're not accurate (which they're not, at least in the case I showed) why is it a good idea to let them escape? So we need them in the calculation, and if it's a nested tree of function calls, they have to travel around too. What do you think is a good place to kill these critters then? Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
[HACKERS] Purpose of rscale/dscale in NUMERIC?
What is the reason for maintaining separate rscale and dscale values in numeric variables? I am finding that this arrangement leads to some odd results, for example this: regression=# select (exp(ln(2.0)) - 2.0); ?column? - -0. (1 row) regression=# select (exp(ln(2.0)) - 2.0) * 10; ?column? - -0.0010 (1 row) The difference between rscale and dscale allows some hidden digits to be carried along in an expression result, and then possibly exposed later. This seems pretty nonintuitive for an allegedly exact calculational datatype. ISTM the policy should be what you see is what you get - no hidden digits. That would mean there's no need for separating rscale and dscale, so I'm wondering why they were put in to begin with. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
