2012/1/25 Tom Diethe <tdie...@gmail.com>: > I originally posted this on stackoverflow - > > http://stackoverflow.com/questions/9002433/how-should-python-dictionaries-be-stored-in-pytables > > so apologies for cross-posting, but maybe this was the better place to > start. > > pytables doesn't natively support python dictionaries. The way I've > approached it is to make a data structure of the form: > > tables_dict = { > 'key' : tables.StringCol(itemsize=40), > > 'value' : tables.Int32Col(), > > } > > (note that I ensure that the keys are <40 characters long) and then create a > table using this structure: > > file_handle.createTable('/', 'dictionary', tables_dict) > > and then populate it with: > > file_handle.dictionary.append(dictionary.items()) > > and retrieve data with: > > dict(file_handle.dictionary.read()) > > This works ok, but reading the dictionary back in is extremely slow. I think > the problem is that the read() function is causing the entire dictionary to > be loaded into memory, which shouldn't really be necessary. Is there a > better way to do this?
Hmm, you may want to use the Table iterator: dict((r['key'], r['value']) for r in file_handle.dictionary.iterrows()) for much better memory usage. Not sure about how much speed you may gain though. -- Francesc Alted ------------------------------------------------------------------------------ Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
