Hello Hans, El dc 24 de 01 del 2007 a les 16:16 +0000, en/na Hans Fangohr va escriure: > With the new information (that arrays don't compress), I have done > some more tests saving just the 'mesh' (as outlined in my last email). > > Reminder of previous findings: > > -680k in ascii format > -110k using 3 CArrays in /mesh > -250k using 3 Arrays in /mesh (which ist just the binary data) > New results: > > -250k written the 3 arrays as one row in table (consistend with line above) > -114k written the 3 arrays as one row in compressed table > > So it turns out that using 3 compressed CArrays gives pretty much the > same compression as writing the 3 arrays (in one row) into a table > (difference is less than 4%, and will possible get smaller for larger > files).
Yeah, I think these figures makes sense. > That's good news, as I can write my main data in a compressed table > (which is what I thought I wanted to do). Generally speaking, the Table objects are the most flexible and powerful in PyTables, so be sure you use them extensively. > Three more question out of curiosity (if I may): > > In this piece of code (where points, simplex and simplexregion are > numpy-arrays): > ---------------------- > filter = tables.Filters(complevel=5,complib='zlib') > > points_chunk = tables.Float64Atom(shape=points.shape) > points_carray = f.createCArray(meshgroup, 'points', points.shape,\ > points_chunk, filters=filter) > points_carray[:] = points > > simplex_chunk = tables.Int32Atom(shape=simplex.shape) > simplex_carray = f.createCArray(meshgroup, 'simplices',simplex.shape,\ > simplex_chunk,filters=filter) > simplex_c_array[:] = simplex > > simplexregion_chunk = tables.Int32Atom(shape=simplexregion.shape) > simplexregion_carray = f.createCArray(meshgroup, 'simplicesregions',\ > simplexregion.shape, > simplexregion_chunk, \ > filters=filter) > simplexregion_carray[:]=simplexregion > ------------------------ > > (i) am I correct to assume that each of the three CArrays is > compressed individually (if you seee what I mean)? Yes. > (ii) if true, and I would like to compress the second and third > CArrays together (because I know that they contain similar data and > should compress well), could I do this somehow using CArrays (if that > question makes sense)? The only way is putting them together as columns of a Table. > (iii) when compressing tables, is there any way to tell the > compression routine about the 'chunk size' to use? Not in 1.x series. Fortunately, you will be able to specify the chunksizes for the 'chunked' objects (i.e. all except Array) in 2.x series. Of course, larger chunksizes will allow you to improve the compression ratio (although not a great deal, I'm afraid). Cheers, -- Francesc Altet | Be careful about using the following code -- Carabos Coop. V. | I've only proven that it works, www.carabos.com | I haven't tested it. -- Donald Knuth ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Pytables-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/pytables-users
