I haven't followed this thread at all so may be off base ... ... but if you're just caring about checking whether the pickled astng is the same, you could just store a sha/md5 of it ...
des. On 28/09/2010, at 21:46 , Emile Anclin wrote: > > Hello, > > I had a quick look trying to see if I can pickle all ASTs. But .. > > On Tuesday 27 July 2010 20:19:21 Edward K. Ream wrote: >>> So the question is, how to store all the data about modified and >>> unmodified files, including a relevant representation of the astng >>> tree? >> >> The ast (or astng) trees would be cached merely to discover that >> special case that the old and new versions of a file differ only by >> comments or whitespace. This is a cute check, but it's optional. > > if I cPickle astng/builder.py using protocol -1, it will take 36 Kb, > for 9 Kb of code, hence a factor of 4. > (protocol 0 will yield 68 Kb) > > Usually, pylint / astng will have to parse most of the Python Standard > Library (which is around 20 Mb) on any bigger project (such as a > framework); plus the site-packages on my current workstation are around > 100 Mb. (rough estimations based on 'du' and considering that the size > of .pyc files are similar to the .py) > > Given that, I am afraid we will have ~ 400 Mb for the only astng dumbs. > So that seems to be the wrong road... > > Edward K Ream, did you have other ideas in mind to handle the problem ? > > -- > > Emile Anclin <[email protected]> > http://www.logilab.fr/ http://www.logilab.org/ > Informatique scientifique & et gestion de connaissances > _______________________________________________ > Python-Projects mailing list > [email protected] > http://lists.logilab.org/mailman/listinfo/python-projects _______________________________________________ Python-Projects mailing list [email protected] http://lists.logilab.org/mailman/listinfo/python-projects
