On May 2, 2018, at 09:42, Gregory Szorc <gregory.sz...@gmail.com> wrote:
> As for things Python could do to make things better, one idea is for "package > bundles." Instead of using .py, .pyc, .so, etc files as separate files on the > filesystem, allow Python packages to be distributed as standalone "archive" > files. Of course, .so files have to be extracted to the file system, because we have to live with dlopen()’s API. In our first release of shiv, we had a loader that did exactly that for just .so files. We ended up just doing .pyz file unpacking unconditionally, ignoring zip-safe, mostly because too many packages still use __file__, which doesn’t work in a zipapp. I’ll plug shiv and importlib.resources (and the standalone importlib_resources) again here. :) > If you go this route, please don't require the use of zlib for file > compression, as zlib is painfully slow compared to alternatives like lz4 and > zstandard. shiv works in a similar manner to pex, although it’s a completely new implementation that doesn’t suffer from huge sys.paths or the use of pkg_resources. shiv + importlib.resources saves us 25-50% of warm cache startup time. That makes things better but still not ideal. Ultimately though that means we don’t suffer from the slowness of zlib since we don’t count cold cache times (i.e. before the initial pyz unpacking operation). Cheers, -Barry
signature.asc
Description: Message signed with OpenPGP
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
