Re: [Python-Dev] Lazy initialization of module global state
On 8 September 2017 at 08:50, Neil Schemenauer wrote: > This should be pretty safe to do and should give a significant > benefit in startup time and memory usage. And given the extended pyc header being proposed in PEP 552, we'd be able to include flags in the header to indicate that the pyc file was compiled with this alternative behaviour. While that wouldn't be needed for import, it would be useful for anyone trying to confirm that the pyc file actually matches the corresponding source file. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Lazy initialization of module global state
This is an idea that came out of the lazy module loading (via AST analysis), posted to python-ideas. The essential idea is to split the marshal data stored in the .pyc into smaller pieces and only load the parts as they are accessed. E.g. use a __getattr__ hook on the module to unmarshal+exec the code. I have a very early prototype: https://github.com/warsaw/lazyimport/blob/master/lazy_compile.py Would work like a "compile_all.py" tool. It writes standard .pyc files right now. It is not there yet but I should use the AST analysis, like the lazy module load stuff, to determine if things have potential side-effects on module import. Those things will get loaded eagerly, like they do now. Initially I was thinking of class definitions and functions but now I realize any global state could get this treatment. E.g. if you have a large dictionary global, don't unmarshal it until someone accesses the module attribute. This should be pretty safe to do and should give a significant benefit in startup time and memory usage. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com