On Mon, 11 Apr 2016 07:33:05 -0400 Donald Stufft <don...@stufft.io> wrote: > > > On Apr 11, 2016, at 7:23 AM, Antoine Pitrou <solip...@pitrou.net> wrote: > > > > On Mon, 11 Apr 2016 07:08:19 -0400 > > Donald Stufft <don...@stufft.io> wrote: > >> > >> I’m not sure if that is still the case with modern SSDs, but I think the > >> idea is that by putting everything inside of zip files you reduce the > >> number of stat calls that Python needs to do (they flip side of this is > >> that pkg_resources is incredibly slow because it needs to issue a ton of > >> stat calls on import). > > > > I don't think SSDs have anything to do in it since the kernel should > > cache directory contents, rather it's the number of system calls issued. > > But as Paul says, in Python 3 importlib got a lot of optimization work > > on this front, so this advice probably doesn't apply anymore. > > > > Surely SSDs are faster at returning the data (metadata or actual data) than > spinning rust and thus would speed up importing on their own. Sure you might > have that data cached in memory if you’ve recently accessed it, but you just > as easily might not have that data cached and the OS might need to hit the > disk to find out.
That doesn't really follow. There is a certain number of disk accesses that is necessary to get the required metadata in a correct way. That number of accesses hasn't changed between Python 2 and Python 3. What has changed is the number of spurious stat() calls asking for information that can be cached. In Python 2, the OS would have done the caching (sparing disk accesses but not system calls). On Python 3, the caching is done inside importlib, which spares system calls in addition to disk accesses. (yes, this is a bit simplified because of NFS and friends ;-)) Regards Antoine. _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
