Re: [basex-talk] sum() is returning inaccurate sums
Am 17.06.2020 um 10:50 schrieb Martin Honnen: Whether you do that with a long `for .. return` or simply with //gross/xs:decimal(.) or //gross!xs:decimal(.) is up to you. Schema-aware XQuery could also help of the `gross` elements were declared as `xs:decimal`, but I don't think XQuery supports schema-aware XQuery. That last sentence was supposed to say: Schema-aware XQuery could also help if the `gross` elements were declared as `xs:decimal`, but I don't think BaseX supports schema-aware XQuery.
Re: [basex-talk] sum() is returning inaccurate sums
Am 17.06.2020 um 10:37 schrieb Jay Straw: Hi all, I'm building a budgeting system for home use, using XSLTForms and -- ready for it? -- BaseX! sum() has been behaving weird. I'm adding USD, which means only 2 decimals of precision. But depending on the combination of elements I add, I'll get a long double that's inaccurate. Sometimes it returns what I expect, sometimes it doesn't, but either case is repeatable. Take these 3 receipts: 1 -- 2020-06-08 Frigid North Jay Debit shop-general 6.50 0.00 6.50 Start Capacitor (for old rockwell saw/jointer motor) 6.50 1 6.50 shop-general 2 -- 2020-06-09 Home Depot Tudor Jay Cash jay-misc 9.87 0 9.87 Stainless Protractor 9.87 1 9.87 jay-misc 3 -- 2018-09-16 Title Wave Midtown Franny Debit books 20.85 20.85 Music of the Spheres books 6.95 1 6.95 Magic Mirror of M. C. Escher 8.95 1 8.95 books Daumier 120 Great Lithographs 4.95 1 4.95 books When I sum the elements: sum 1 & 2 = 16.367 sum 1 & 3 = 27.35 sum 2 & 3 = 30.72 What makes certain combinations return such crazy numbers? I did find one other thread in the mailing list about this, it got an answer but didn't really explain much. I have a schema for prices in the XForm, should I be loading that somehow? The only workaround I have now is like, "let $grosses:= for $gross in $receipts//gross return xs:decimal($gross)" but I'm hoping there's a more straightforward way. Not sure if this is implementation specific or if it's not defined or what. If I should ask elsewhere, please let me know :-) The "default" xs:double data type used in XPath and XQuery is known to have such inaccuracies so (try the same sum in JavaScript and you should see similar results), yes, you need to use `xs:decimal` explicitly to avoid such problems. Whether you do that with a long `for .. return` or simply with //gross/xs:decimal(.) or //gross!xs:decimal(.) is up to you. Schema-aware XQuery could also help of the `gross` elements were declared as `xs:decimal`, but I don't think XQuery supports schema-aware XQuery.
[basex-talk] sum() is returning inaccurate sums
Hi all, I'm building a budgeting system for home use, using XSLTForms and -- ready for it? -- BaseX! sum() has been behaving weird. I'm adding USD, which means only 2 decimals of precision. But depending on the combination of elements I add, I'll get a long double that's inaccurate. Sometimes it returns what I expect, sometimes it doesn't, but either case is repeatable. Take these 3 receipts: 1 -- 2020-06-08 Frigid North Jay Debit shop-general 6.50 0.00 6.50 Start Capacitor (for old rockwell saw/jointer motor) 6.50 1 6.50 shop-general 2 -- 2020-06-09 Home Depot Tudor Jay Cash jay-misc 9.87 0 9.87 Stainless Protractor 9.87 1 9.87 jay-misc 3 -- 2018-09-16 Title Wave Midtown Franny Debit books 20.85 20.85 Music of the Spheres books 6.95 1 6.95 Magic Mirror of M. C. Escher 8.95 1 8.95 books Daumier 120 Great Lithographs 4.95 1 4.95 books When I sum the elements: sum 1 & 2 = 16.367 sum 1 & 3 = 27.35 sum 2 & 3 = 30.72 What makes certain combinations return such crazy numbers? I did find one other thread in the mailing list about this, it got an answer but didn't really explain much. I have a schema for prices in the XForm, should I be loading that somehow? The only workaround I have now is like, "let $grosses:= for $gross in $receipts//gross return xs:decimal($gross)" but I'm hoping there's a more straightforward way. Not sure if this is implementation specific or if it's not defined or what. If I should ask elsewhere, please let me know :-) Be well, Jay
