El dv 08 de 12 del 2006 a les 07:00 -0800, en/na Stefan Kuzminski va escriure: > Hi, > > I'm using compression normally on my .h5 files. However the resulting files > are still 'too' large ( when compared to csv.gz for example ). I tried a > number of compression settings and compressors, with marginal improvements. > > However, if I go and actually gzip the .h5 file, I can radically reduce the > size of the file.. > > Here is a .h5 created with complevel=5 ( 1,000x3,000 Float32s ) > somedata.h5 -> 1,290,833 ( 1.2 meg ) > > Now I gzip it, and it practically disappears! > somedata.h5.gz -> 16,345 ( 0.016 meg ) > > For reference, the csv.gz of the same data is.. > somedata.csv.gz -> 213,064 ( .21 meg ) > > My question is, is there a way I can get this kind of compression 'in > stream'? And why does the regular compression seem to compare poorly > to the csv.gz. The data are integers > 1,000,000.
Mmm, that's strange. Which kind of leaves are you using for your data? Using CArrays with a very small chunksize could lead to this. EArrays with a small rowsize can also expose this behaviour. Can you send a small example of your code so that I can have a look at it and try to suggest some improvement? Cheers, -- Francesc Altet | Be careful about using the following code -- Carabos Coop. V. | I've only proven that it works, www.carabos.com | I haven't tested it. -- Donald Knuth ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
