On Wed, Oct 27, 2010 at 20:07, Francesc Alted <fal...@pytables.org> wrote: > A Wednesday 27 October 2010 15:38:28 Gaetan de Menten escrigué:
>> [table.copy() and friends are awfully slow with many columns] > After investigating this, I come to the conclusion that the overhead > comes from PyTables when copying a couple of attributes per column > (namely FIELD_N_NAME and FIELD_N_FILL, where N is the column number). > > I suspect that the ultimate responsible is an inefficiency in the HDF5 > for dealing with these attributes (I should investigate more, though), > so meanwhile I decided not copy the attributes during `Table.copy()` > operations. With this, performance is good now. More info: > > http://pytables.org/trac/ticket/304 Wow. Thanks a lot for the quick reaction. > Anyway, I'm a bit fed up with such FIELD_N_NAME and FIELD_N_FILL > attributes that are not really useful (except for some rare cases). So > I'm thinking in removing them completely for PyTables 2.3, see: > > http://pytables.org/trac/ticket/305 > > If anyone is against this, please speak now or forever hold your peace! > (I'll announce this in a proper thread also). I'm not against this, but that's mostly because I don't know hdf5 enough to know what this implies. All I ask is that the files produced by PyTables stay readable by other tools. -- Gaëtan de Menten ------------------------------------------------------------------------------ Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
