Hi Francesc: Many thanks for your great feedback. Based on your suggestion, I may rethink my earlier approach. I tried using compression at one point, and it did reduce the file size by approximately half. I had (naively) assumed that by using compression and a float16 type I would cut file size by another half. However, it sounds like you might be suggesting that the compression was particularly efficient because the data did not require 32 bits of precision.
One other approach I considered was simply taking the numpy arrays and reinterpreting the bits as 32-bit (e.g. using the ndarray.view() method) before appending to the EArray. Brad On Sun, Jun 12, 2011 at 6:24 AM, Francesc Alted <fal...@pytables.org> wrote: > A Friday 10 June 2011 22:18:04 Brad Buran escrigué: >> What is the simplest way to implement a Float16Atom for PyTables? I >> ran a few quick tests using Numpy 1.6 and the latest version of >> PyTables and it did not work out of the box. I even edited >> tables/atom.py to include float16 but it does not appear that adding >> a Float16Atom is enough. The error came from the library created by >> utilsExtension.pyx. At this point, I got a bit stumped. I'm >> willing to push deeper into the code if it's a relatively simple >> task of adding the right definitions in the right places. However, >> before I do that, I want to make sure that this hasn't already been >> done before, and if it hasn't been done before, whether there are >> any larger issues to adding a Float16Atom to PyTables that I may be >> ignorant of. > > First of all, you need to implement this new type with the underlying > HDF5 API. See: > > http://hdf-forum.184993.n3.nabble.com/Half-precision-floating-point- > format-td194774.html > > for hints on how to do this. > > Next, it is just a matter of adding this new type to > tables/utilsExtension.pyx, and then allow for a 'float16' dtype in > tables/atom.py (and declare it in tables/__init__.py). I think that > should be all. > > While the above should be feasible, I much easier way to make 'float16' > types taking less space on-disk is to use 'float32' in combination with > compression. > > Cheers, > > -- > Francesc Alted > > ------------------------------------------------------------------------------ > EditLive Enterprise is the world's most technically advanced content > authoring tool. Experience the power of Track Changes, Inline Image > Editing and ensure content is compliant with Accessibility Checking. > http://p.sf.net/sfu/ephox-dev2dev > _______________________________________________ > Pytables-users mailing list > Pytables-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/pytables-users > ------------------------------------------------------------------------------ EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
