Hi David, A Monday 18 May 2009 00:40:46 David Warde-Farley escrigué: > Hi there, > > I am switching to using PyTables for some image-processing work to > store some intermediate data, namely binary masks, in a structured/ > hierarchical way. > > I've got about 182MB worth of binary PNG files, and when I store them > as CArrays, even with zlib or bzip2 complevel=9, they balloon to > 400MB. This isn't a huge deal (disk space is cheap and I can always > just compress the h5 file afterward if I need to transfer it > somewhere, that seems to almost cut the file size in half) but does > anyone have any tips for storing large arrays of BoolAtoms efficiently?
Mmh, looks like the HDF5 N-filter could be of help here: http://www.hdfgroup.org/HDF5/doc/UG/10_Datasets.html#N-Bit Unfortunately, this has not been implemented in PyTables yet (contributions are welcome!). However, I can't understand well why a compression of the entire file, made of already compressed CArrays, can cut the size in half. Mmh, could you please provide a script (with synthetic data) showing that? If this is not easy, could provide a sample file in some public server? Cheers, -- Francesc Alted ------------------------------------------------------------------------------ Crystal Reports - New Free Runtime and 30 Day Trial Check out the new simplified licensing option that enables unlimited royalty-free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
