The number of digits for decimal is larger than float or double. Thats the problem. The implementation internally for decimal division converts to double, divides then converts back to decimal in the process losing full decimal precision but displaying the closest value. If you convert the result to a double you get the rounding back to 72. xs:double( 180 div 2.5) == 72
Since precision of decimal is system defined this isnt non-conformant, but I agree it's not ideal. A simple case to demonstrate this is an even less prety 5 div 2.5 => 1.999999999999999889 xs:double( 5 div 2.5 ) => 2 ----------------------------------------------------------------------------- David Lee Lead Engineer MarkLogic Corporation [email protected] Phone: +1 812-482-5224 Cell: +1 812-630-7622 www.marklogic.com<http://www.marklogic.com/> From: [email protected] [mailto:[email protected]] On Behalf Of Alex Milowski Sent: Wednesday, March 20, 2013 11:53 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Simple Math Issue It still feels like a bug in the underlying decimal implementation. I'm having a hard time imagining a world where "180 / 2.5" is a hard computation to get right. The number of digits of precision has to be greater than 3 (or 4). Appendix E of the XPath F&O lists the number of digits of precision for xs:decimal as implementation defined. As both operands are xs:decimal and so there is no type promotion to xs:double. As such, I expect the following computation to be performed: 180 / 2.5 = 1800 / 25 = 72 [2]. [1] http://www.w3.org/TR/xpath-functions/ [2] http://en.wikipedia.org/wiki/Fixed_point_arithmetic On Wed, Mar 20, 2013 at 7:36 AM, David Lee <[email protected]<mailto:[email protected]>> wrote: The difference here is that 2.5 is a xs:decimal not an xs:float or xs:double according to XQuery http://www.w3.org/TR/xquery/#prod-xquery-DecimalLiteral Decimal types are different precision from float or double and are not floating point. Neither of which is infinite precision. So math sometimes doesn't produce an exact mathematically correct result. Yes I know that doesn't make the answer any more pleasant ----------------------------------------------------------------------------- David Lee Lead Engineer MarkLogic Corporation [email protected]<mailto:[email protected]> Phone: +1 812-482-5224<tel:%2B1%20812-482-5224> Cell: +1 812-630-7622<tel:%2B1%20812-630-7622> www.marklogic.com<http://www.marklogic.com> -----Original Message----- From: [email protected]<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] On Behalf Of David Sewell Sent: Wednesday, March 20, 2013 9:36 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Simple Math Issue I get the same results as Alex running version 6.0-2.1 under Linux. On Wed, 20 Mar 2013, Geert Josten wrote: > Which version of MarkLogic are you running exactly? I am running 6.0-1.1 on > win7 and get (72, 72, 72) as response.. > > > > Kind regards, > > Geert > > > > *Van:* > [email protected]<mailto:[email protected]> > [mailto: > [email protected]<mailto:[email protected]>] > *Namens *Alex Milowski > *Verzonden:* dinsdag 19 maart 2013 23:09 > *Aan:* General Mark Logic Developer Discussion > *Onderwerp:* [MarkLogic Dev General] Simple Math Issue > > > > In my Mathematical mind, the following expression: > > > > (180 div 2.5, 180 div xs:float(2.5), 180 div xs:double(2.5)) > > > > should yeild: > > > > (72, 72, 72) > > > > as the correct answer, mathematically speaking, is 72. > > > > MarkLogic 6 seems to think it is: > > > > (71.99999999999999601, 72, 72) > > > > which leads me to believe this is a xs:decimal issue. > > > > > -- David Sewell, Editorial and Technical Manager ROTUNDA, The University of Virginia Press PO Box 400314, Charlottesville, VA 22904-4314 USA Email: [email protected]<mailto:[email protected]> Tel: +1 434 924 9973<tel:%2B1%20434%20924%209973> Web: http://rotunda.upress.virginia.edu/ _______________________________________________ General mailing list [email protected]<mailto:[email protected]> http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected]<mailto:[email protected]> http://developer.marklogic.com/mailman/listinfo/general -- --Alex Milowski "The excellence of grammar as a guide is proportional to the paucity of the inflexions, i.e. to the degree of analysis effected by the language considered." Bertrand Russell in a footnote of Principles of Mathematics
_______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
