This is an idea that came out of the lazy module loading (via AST analysis), posted to python-ideas. The essential idea is to split the marshal data stored in the .pyc into smaller pieces and only load the parts as they are accessed. E.g. use a __getattr__ hook on the module to unmarshal+exec the code.
I have a very early prototype: https://github.com/warsaw/lazyimport/blob/master/lazy_compile.py Would work like a "compile_all.py" tool. It writes standard .pyc files right now. It is not there yet but I should use the AST analysis, like the lazy module load stuff, to determine if things have potential side-effects on module import. Those things will get loaded eagerly, like they do now. Initially I was thinking of class definitions and functions but now I realize any global state could get this treatment. E.g. if you have a large dictionary global, don't unmarshal it until someone accesses the module attribute. This should be pretty safe to do and should give a significant benefit in startup time and memory usage. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
