Am 17.06.2020 um 10:37 schrieb Jay Straw:
Hi all,
I'm building a budgeting system for home use, using XSLTForms and --
ready for it? -- BaseX!
sum() has been behaving weird. I'm adding USD, which means only 2
decimals of precision. But depending on the combination of elements I
add, I'll get a long double that's inaccurate. Sometimes it returns what
I expect, sometimes it doesn't, but either case is repeatable.
Take these 3 receipts:
1 --
<receipt uuid="694bf777-2d40-42ad-9809-97dfe6b04f40">
<metadata>
<date>2020-06-08</date>
<store>
<name>Frigid North</name>
<location/>
</store>
<purchaser>Jay</purchaser>
<account>Debit</account>
<category>shop-general</category>
<net-total-computed>6.50</net-total-computed>
<tip>0.00</tip>
<gross>6.50</gross>
</metadata>
<item>
<name>Start Capacitor (for old rockwell saw/jointer motor)</name>
<cost>6.50</cost>
<qty>1</qty>
<total>6.50</total>
<category>shop-general</category>
</item>
</receipt>
2 --
<receipt uuid="2eb74e82-3db5-4bd9-a591-8621c79f3fd6">
<metadata>
<date>2020-06-09</date>
<store>
<name>Home Depot</name>
<location>Tudor</location>
</store>
<purchaser>Jay</purchaser>
<account>Cash</account>
<category>jay-misc</category>
<net-total-computed>9.87</net-total-computed>
<tip>0</tip>
<gross>9.87</gross>
</metadata>
<item>
<name>Stainless Protractor</name>
<cost>9.87</cost>
<qty>1</qty>
<total>9.87</total>
<category>jay-misc</category>
</item>
</receipt>
3 --
<receipt uuid="1f1462e6-b37f-4daa-b293-29b9ec353627">
<metadata>
<date>2018-09-16</date>
<store>
<name>Title Wave</name>
<location>Midtown</location>
</store>
<purchaser>Franny</purchaser>
<account>Debit</account>
<category>books</category>
<net-total-computed>20.85</net-total-computed>
<tip/>
<gross>20.85</gross>
</metadata>
<item>
<name>Music of the Spheres</name>
<category>books</category>
<cost>6.95</cost>
<qty>1</qty>
<total>6.95</total>
</item>
<item>
<name>Magic Mirror of M. C. Escher</name>
<cost>8.95</cost>
<qty>1</qty>
<total>8.95</total>
<category>books</category>
</item>
<item>
<name>Daumier 120 Great Lithographs</name>
<cost>4.95</cost>
<qty>1</qty>
<total>4.95</total>
<category>books</category>
</item>
</receipt>
When I sum the <gross/> elements:
sum 1 & 2 = 16.369999999999997
sum 1 & 3 = 27.35
sum 2 & 3 = 30.72
What makes certain combinations return such crazy numbers?
I did find one other thread in the mailing list about this, it got an
answer but didn't really explain much. I have a schema for prices in the
XForm, should I be loading that somehow? The only workaround I have now
is like, "let $grosses:= for $gross in $receipts//gross return
xs:decimal($gross)" but I'm hoping there's a more straightforward way.
Not sure if this is implementation specific or if it's not defined or
what. If I should ask elsewhere, please let me know :-)
The "default" xs:double data type used in XPath and XQuery is known to
have such inaccuracies so (try the same sum in JavaScript and you should
see similar results), yes, you need to use `xs:decimal` explicitly to
avoid such problems.
Whether you do that with a long `for .. return` or simply with
//gross/xs:decimal(.)
or
//gross!xs:decimal(.)
is up to you.
Schema-aware XQuery could also help of the `gross` elements were
declared as `xs:decimal`, but I don't think XQuery supports schema-aware
XQuery.
