On Wed, Jul 14, 2010 at 05:15, Nick Coghlan <ncogh...@gmail.com> wrote:
> On Wed, Jul 14, 2010 at 9:05 PM, Steve Holden <st...@holdenweb.com> wrote: > >> I have stopped fixing bugs related to this in import.c because of the > >> annoying issues it causes and I expect the correct approach to gain > >> traction at some point (plus get importlib bootstrapped in so I don't > >> have to care about import.c anymore). > >> > > It's only a matter of time until someone decides to provide a C > > implementation of importlib ;-) > > A C accelerated version of importlib would probably be an awful lot > cleaner than the current import implementation. While the import code > isn't quite the mess that we sometimes make it out to be (it does > basically work after all, and most of the "problems" lie in dim dark > corners that 99% of developers will never get close to), but it has > definitely suffered from an accumulation of features on top of a core > approach that has been pushed far beyond what it was originally > designed to support. > So my dream is to finally get full compatibility for importlib in 3.3 (probably won't hit 3.2 as it requires me changing marshal.loads to take a file path argument) and then try to bootstrap it in. Now bootstrapping can be done with actually a minimal amount of C code as I can simply make the bytecode for importlib a literal in C, get that loaded, and then import importlib as found on the file system to allow people to override things. But obviously I could also identify the true bottlenecks through profiling and provide acceleration where useful. Trick is being reasonable about this so as to not put other VMs at a disadvantage by making the bootstrap solution too difficult to implement. > > That said, I believe the limiting factor in import speed is likely to > remain the number of stat calls and other filesystem operations, so it > will be interesting to find out just how significant a slowdown there > is between import.c and importlib. If I'm right about the real source > of bottlenecks in import performance, the difference may be > surprisingly small. So I started writing benchmark code in anticipation of needing to prove a minimal performance difference to justify bootstrapping importlib. Right now it only compares importing from sys.modules and built-in modules. You can run it with ``./python.exe -m importlib.test.benchmark``. If you add a `-b` option that will use the built-in __import__ implementation. I still need to benchmark loading source, bytecode, writing bytecode, and loading extensions. Maybe I will finish writing the benchmark code as the thing I do while at EuroPython (that and finally getting to reviewing http://bugs.python.org/issue2919 so that cProfile and profile can merge, unless someone beats me to it, in which case I would be grateful =).
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
