Hi Peter,
Reading through issue #91 in trac:
"While Python's decimal package provides exact values, which is good, it
also incorporates a concept of significant digits that is not present in
XML's xs:decimal, which may not be good."
Could you clarify this one for me? The XML Schema spec says:
"Note: All minimally conforming processors must support decimal numbers
with a minimum of 18 decimal digits"
It looks to me that my Python 2.7 installation does 28 decimal digits by
default:
>>> aa = decimal.Decimal("0.12345678901234567")
>>> print aa
0.12345678901234567
>>> bb = decimal.Decimal("123456789012345678.0")
>>> print bb
123456789012345678.0
>>> aa = decimal.Decimal("0.123456789012345678")
>>> print aa
0.123456789012345678
>>> print aa + bb
123456789012345678.1234567890
>>> decimal.getcontext().prec
28
Am I missing something here? I'm interpreting "decimal digits" in the XML
Schema spec to be defined the same as "significant digits" in the Python
documentation. It looks, from the "print aa + bb" that the two terms are
equivalent.
I think a follow on question to before is "what type does the XML Schema
specification suggest we use for money" (where precision and rounding
issues in monetary calculations are unacceptable)? Some data storage
mechanisms have specific money types (PostgreSQL's MONEY type comes to
mind, although NUMERIC(x,y) is a more portable way of achieving that). The
XML Schema spec includes an example of monetary value storage in
http://www.w3.org/TR/xmlschema-0/#complexTfromSimpleT :
<internationalPrice currency="EUR">423.46</internationalPrice>
Defined as:
<xsd:element name="internationalPrice">
<xsd:complexType>
<xsd:simpleContent>
<xsd:extension base="xsd:decimal">
<xsd:attribute name="currency" type="xsd:string"/>
</xsd:extension>
</xsd:simpleContent>
</xsd:complexType>
</xsd:element>
So it is indeed the intention that xsd:decimal be used as the money type.
In the mailing list thread referenced in Trac #91, it claims "The problem
is that this patch will return a Decimal type and some operations aren't
allowed like multiplication". Is this trying to say that multiplying two
decimals together? Because:
nathanr@gionta:~> python
Python 2.7.3 (default, Apr 14 2012, 08:58:41) [GCC] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import decimal
>>> a = decimal.Decimal("1.5")
>>> b = decimal.Decimal("2.5")
>>> type(a)
<class 'decimal.Decimal'>
>>> type(b)
<class 'decimal.Decimal'>
>>> c = a*b
>>> print c
3.75
>>> type(c)
<class 'decimal.Decimal'>
>>>
that doesn't seem to be the case. Sure, this fails:
>>> x = decimal.Decimal("2.5")
>>> type(x)
<class 'decimal.Decimal'>
>>> x * 2.3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for *: 'Decimal' and 'float'
>>>
but I find that acceptable, in fact reassuring, as in many cases that'll be
a bug. If not, Python 2.7 adds a new constructor for Decimal, allowing it
to take a float so it's really easy to say "yes, I really mean to multiply
my nice tidy arbitrary precision decimal with that float":
>>> import decimal
>>> x = decimal.Decimal("2.5")
>>> y = 2.5
>>> type(y)
<type 'float'>
>>> print x * decimal.Decimal(y)
6.25
>>> z = x * decimal.Decimal(y)
>>> print z
6.25
>>> type(z)
<class 'decimal.Decimal'>
>>>
And I actually think that's the right way. People who explicitly want
floats should use xsd:float. Those who need the guarantees that xsd:decimal
was designed to provide get it (as Java provides in JAXB), and the platform
actually helps stop you from accidentally mixing the two up, but provides
an easy way to interoperate if you need to. I'd be very surprised if there
were any users who relied on (and are insisting on) xsd:decimal mapping to
float, and if they are they can just change their schema declarations to
xsd:float and be otherwise unaffected (zero code change). I'd say there are
a whole lot more people who, like me, are blindly assuming that xsd:decimal
and xsd:float mean two different things.
However, given this is unlikely to change in the near future, and it is for
us a production issue, I've just asked one of our engineers to change all
our xsd:decimals to xsd:string, change the code that uses it, and do a full
code audit afterwards. Our XML Schema uses xsd:decimal for monetary
amounts, and we do calculations based off that, so precision issues are
important to us. If in the future PyXB changes its mapping of xsd:decimal
to something that is arbitrary precision we'll port our code back to
xsd:decimal.
Thanks for looking at this, and the quick response.
Regards,
Nathan.
On 13 March 2013 13:26, Peter Bigot <big...@acm.org> wrote:
> This issue also exists as:
> https://sourceforge.net/apps/trac/pyxb/ticket/91
>
> Based on a very quick review of the comments in that ticket, as well as
> the original discussion that it references, it is unlikely that the base
> type underlying the decimal type in PyXB will change because there seem to
> be a couple incompatibilities between Python decimal and XML Schema
> decimal, including the effect of significant digits and lack of support for
> certain operators.
>
> I'd have to spend some time analyzing all that to determine what the
> correct change would be, informed by how people actually use xs:decimal as
> opposed to xs:float. It might be something to revisit if/when there's a
> Python3 update to PyXB, but don't count on it happening spontaneously any
> time soon.
>
> Peter
>
>
> On Tue, Mar 12, 2013 at 8:28 PM, Nathan Robertson <nath...@nathanr.net>wrote:
>
>> Hi,
>>
>> I was just looking at the PyXB API documentation. Is there any difference
>> in the PyXB implementation of xsd:float from xsd:decimal? Looking at:
>>
>> http://pyxb.sourceforge.net/api/pyxb.binding.datatypes.decimal-class.html
>>
>> it says that it inherits from the Python float type, which is an IEEE
>> binary floating point number. The XML Schema specification says that
>> xsd:float should be a 32 bit IEEE floating point number, but it seems stops
>> short on defining xsd:decimal as arbitrary precision - it just says at
>> least 18 decimal digits must be supported. But it is certainly true that
>> the way they define and describe xsd:float vs. xsd:decimal are quite
>> different.
>>
>> (http://www.w3.org/TR/xmlschema-2/#decimal)
>>
>> I note in the PyXB documentation it says:
>>
>> "*To Do:* The Python base type for this is wrong. Consider
>> http://code.google.com/p/mpmath/";.
>>
>> Is there any reason why the base python type shouldn't just be the built
>> in decimal type? From what I gather, this is arbitrary precision. As a
>> point of note, JAXB uses java.lang.BigDecimal (arbitrary precision floating
>> point number) for xsd:decimal.
>>
>> Traditionally, we (as in, at my workplace) have used xsd:decimal for
>> monetary values in our XML documents, and obviously want to ensure that
>> under no circumstances these get rounded. I'm thinking of changing all the
>> types in my XML schema to xsd:string to avoid this type mapping issue, and
>> manually converting the string to an arbitrary precision decimal type
>> (probably the inbuilt python decimal type). But before I do that, is there
>> any chance that the type that PyXB uses can be or will be changed from
>> being based on an IEEE floating point number (with it's rounding and
>> precision issues) to an arbitrary precision decimal type of some sort?
>> Doing so seems to make sense - if the user really wants IEEE floating
>> point, xsd:float is the XML Schema type for that. And the documentation
>> seems to suggest that this was the thinking, at least at some point in the
>> past.
>>
>> Or am I missing something here?
>>
>> Regards,
>> Nathan.
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Everyone hates slow websites. So do we.
>> Make your web apps faster with AppDynamics
>> Download AppDynamics Lite for free today:
>> http://p.sf.net/sfu/appdyn_d2d_mar
>> _______________________________________________
>> pyxb-users mailing list
>> pyxb-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/pyxb-users
>>
>>
>
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
pyxb-users mailing list
pyxb-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pyxb-users
