This has been sent to the PyTables users' list, but bounced. Please, remember to always subscribe to the list before sending messages.
Thanks, ---------- Missatge transmès ---------- Subject: Float32 accuracy Date: Sunday 02 March 2008 From: "Christiaan Putter" <[EMAIL PROTECTED]> To: pytables-users@lists.sourceforge.net Hi guys and girls, I'm having some strange inaccuracies when storing data into float32 columns in a table. I'm storing historical financial data so I'm rounding most things to 2 decimal places. Which means the values I write into tables tend to look like 107.43000000000001 or 110.31999999999999 Which is the 'normal' inaccuracy I'd expect from floating point values in python ( double precision ?) So after writing 37.14 to my table I'd ge 37.1399993896 back. So this is the normal single precision inaccuracy I'd expect. The problem: Storing 12266.39 gives me 12266.3896484 Normally I'd expect 7 digits to be accurate when working with single precision floats, though here only 6 digits are. Am I wrong to assume 7 digits to be accurate or is something else behind this? Clearly working with float64 would solve this, though it greatly increases the size of my hdf5 files. Any comments or suggestions would be appreciated. Have a nice day, cputter ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
