I haven't used Python's Decimal module but you might find it helpful:
http://docs.python.org/lib/module-decimal.html
Kevin Christman
----- Original Message ----
From: Francesc Altet <[EMAIL PROTECTED]>
To: pytables-users@lists.sourceforge.net
Sent: Monday, March 3, 2008 2:25:42 AM
Subject: [Pytables-users] Fwd: Float32 accuracy
This has been sent to the PyTables users' list, but bounced. Please,
remember to always subscribe to the list before sending messages.
Thanks,
---------- Missatge transmès ----------
Subject: Float32 accuracy
Date: Sunday 02 March 2008
From: "Christiaan Putter" <[EMAIL PROTECTED]>
To: pytables-users@lists.sourceforge.net
Hi guys and girls,
I'm having some strange inaccuracies when storing data into float32
columns in a table.
I'm storing historical financial data so I'm rounding most things to 2
decimal places. Which means the values I write into tables tend to look
like
107.43000000000001 or 110.31999999999999
Which is the 'normal' inaccuracy I'd expect from floating point values
in python ( double precision ?)
So after writing 37.14 to my table I'd ge 37.1399993896 back. So this
is the normal single precision inaccuracy I'd expect.
The problem:
Storing 12266.39 gives me 12266.3896484
Normally I'd expect 7 digits to be accurate when working with single
precision floats, though here only 6 digits are.
Am I wrong to assume 7 digits to be accurate or is something else behind
this?
Clearly working with float64 would solve this, though it greatly
increases the size of my hdf5 files.
Any comments or suggestions would be appreciated.
Have a nice day,
cputter
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
____________________________________________________________________________________
Never miss a thing. Make Yahoo your home page.
http://www.yahoo.com/r/hs-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
