On 10/29/2011 01:08 AM, Roy Smith wrote: > In article <mailman.2293.1319834877.27778.python-l...@python.org>, > Gelonida N <gelon...@gmail.com> wrote: > >> I would like to save many dicts with a fixed amount of keys >> tuples to a file in a memory efficient manner (no random, but only >> sequential access is required) > > There's two possible scenarios here. One, which you seem to be > exploring, is to carefully study your data and figure out the best way > to externalize it which reduces volume. > > The other is to just write it out in whatever form is most convenient > (JSON is a reasonable thing to try first), and compress the output. Let > the compression algorithms worry about extracting the entropy. You may > be surprised at how well it works. It's also an easy experiment to try, > so if it doesn't work well, at least it didn't cost you much to find out.
Yes I have to make some more tests to see the defference between just compressing aplain format (JSON / pickle) and compressing the 'optimized' representation. -- http://mail.python.org/mailman/listinfo/python-list
