A Monday 13 June 2011 07:25:18 Brad Buran escrigué: > Hi Francesc: > > Many thanks for your great feedback. Based on your suggestion, I may > rethink my earlier approach. I tried using compression at one point, > and it did reduce the file size by approximately half. I had > (naively) assumed that by using compression and a float16 type I > would cut file size by another half. However, it sounds like you > might be suggesting that the compression was particularly efficient > because the data did not require 32 bits of precision.
Well, it depends on the entropy of your datasets, but I'd say that if you were getting a 2x compression ratio it is mainly due to your floats do not filling the complete range of float32 type. Of course, it is better to have native support for float16, but IMO, most of the cases is enough with compression. > One other approach I considered was simply taking the numpy arrays > and reinterpreting the bits as 32-bit (e.g. using the ndarray.view() > method) before appending to the EArray. Exactly. And the opposite: reinterpret float32 as float16 during reading. -- Francesc Alted ------------------------------------------------------------------------------ EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
