Re: [Python-Dev] importlib hooray!

On Wed, Feb 6, 2013 at 4:26 AM, Thomas Heller wrote: > I have become a fan of the new python 3.3 importlib > in the last few days. > Glad it's working out for you! > > It has allowed me to write a ModuleMapper which I put into > sys.metapath (in sitecustomize.py, for Python 3.3). > > This mapp

[Python-Dev] importlib hooray!

I have become a fan of the new python 3.3 importlib in the last few days. It has allowed me to write a ModuleMapper which I put into sys.metapath (in sitecustomize.py, for Python 3.3). This mapper currently does rename modules like 'Queue' or '_winreg' to the Python3 modules 'queue' or 'winreg'

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On 04/20/2012 10:59 AM, Brett Cannon wrote: > > > On Fri, Apr 20, 2012 at 09:54, Eric V. Smith > wrote: > > On 04/14/2012 02:12 PM, Brett Cannon wrote: > > My multi-year project -- started in 2006 according to my blog -- to > > rewrite import in pure Pytho

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Fri, Apr 20, 2012 at 11:02, Eric Snow wrote: > On Fri, Apr 20, 2012 at 7:54 AM, Eric V. Smith wrote: > > On 04/14/2012 02:12 PM, Brett Cannon wrote: > >> My multi-year project -- started in 2006 according to my blog -- to > >> rewrite import in pure Python and then bootstrap it into CPython as

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Fri, Apr 20, 2012 at 7:54 AM, Eric V. Smith wrote: > On 04/14/2012 02:12 PM, Brett Cannon wrote: >> My multi-year project -- started in 2006 according to my blog -- to >> rewrite import in pure Python and then bootstrap it into CPython as >> *the* implementation of __import__() is finally over

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Fri, Apr 20, 2012 at 09:54, Eric V. Smith wrote: > On 04/14/2012 02:12 PM, Brett Cannon wrote: > > My multi-year project -- started in 2006 according to my blog -- to > > rewrite import in pure Python and then bootstrap it into CPython as > > *the* implementation of __import__() is finally ove

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On 04/14/2012 02:12 PM, Brett Cannon wrote: > My multi-year project -- started in 2006 according to my blog -- to > rewrite import in pure Python and then bootstrap it into CPython as > *the* implementation of __import__() is finally over (mostly)! Maybe I'm missing something, but it seems that I

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Tue, Apr 17, 2012 at 13:39, Antoine Pitrou wrote: > On Tue, 17 Apr 2012 11:41:32 -0400 > Brett Cannon wrote: > > > > Actually Cython would help with a subtle maintenance burden of > maintaining > > *any* C code for import. Right now, > > Python/import.c:PyImport_ImportModuleLevelObject() is a

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Tue, 17 Apr 2012 11:41:32 -0400 Brett Cannon wrote: > > Actually Cython would help with a subtle maintenance burden of maintaining > *any* C code for import. Right now, > Python/import.c:PyImport_ImportModuleLevelObject() is an accelerated C > version of importlib.__import__() through checking

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Tue, Apr 17, 2012 at 06:43, Eric V. Smith wrote: > On 4/17/2012 5:52 AM, Antoine Pitrou wrote: > > On Mon, 16 Apr 2012 20:41:56 -0400 > > Brett Cannon wrote: > >> On Mon, Apr 16, 2012 at 20:27, Antoine Pitrou > wrote: > >> > >>> On Tue, 17 Apr 2012 01:11:14 +0200 > >>> Georg Brandl wrote: >

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Tue, 17 Apr 2012 01:11:14 +0200, Georg Brandl wrote: > On 16.04.2012 18:15, R. David Murray wrote: > > I don't see how depending on Cython is better than depending on having > > an existing Python. > > No, it's not just an existing Python, it is (at least currently) the same > version of Pytho

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On 4/17/2012 5:52 AM, Antoine Pitrou wrote: > On Mon, 16 Apr 2012 20:41:56 -0400 > Brett Cannon wrote: >> On Mon, Apr 16, 2012 at 20:27, Antoine Pitrou wrote: >> >>> On Tue, 17 Apr 2012 01:11:14 +0200 >>> Georg Brandl wrote: No, it's not just an existing Python, it is (at least current

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Mon, 16 Apr 2012 20:41:56 -0400 Brett Cannon wrote: > On Mon, Apr 16, 2012 at 20:27, Antoine Pitrou wrote: > > > On Tue, 17 Apr 2012 01:11:14 +0200 > > Georg Brandl wrote: > > > > > > No, it's not just an existing Python, it is (at least currently) the same > > > version of Python being buil

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

Antoine Pitrou, 17.04.2012 02:27: > On Tue, 17 Apr 2012 01:11:14 +0200 > Georg Brandl wrote: >> No, it's not just an existing Python, it is (at least currently) the same >> version of Python being built. Therefore I wrote about the bootstrapping >> problems when bytecode changes. >> >> Depending o

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

Amaury Forgeot d'Arc, 16.04.2012 22:43: > 2012/4/16 Stefan Behnel >>> On the down side, we would depend on Cython for evolving . >> >> Right, although not as a strict dependency. The code would still work just >> fine in plain Python. > > Not quite, we are talking of the imp module here... Hmm, r

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Mon, Apr 16, 2012 at 20:27, Antoine Pitrou wrote: > On Tue, 17 Apr 2012 01:11:14 +0200 > Georg Brandl wrote: > > > > No, it's not just an existing Python, it is (at least currently) the same > > version of Python being built. Therefore I wrote about the bootstrapping > > problems when byteco

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Tue, 17 Apr 2012 01:11:14 +0200 Georg Brandl wrote: > > No, it's not just an existing Python, it is (at least currently) the same > version of Python being built. Therefore I wrote about the bootstrapping > problems when bytecode changes. > > Depending on Cython is better in that it breaks t

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On 16.04.2012 18:15, R. David Murray wrote: On Mon, 16 Apr 2012 11:21:34 -0400, Brett Cannon wrote: On Mon, Apr 16, 2012 at 10:07, "Martin v. Löwis" wrote: > > We have other instances of this (e.g. the Objects/typeslots.inc file > > is generated and checked in), but in the case of imp

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

Am 16.04.2012 19:44, schrieb Antoine Pitrou: > On Mon, 16 Apr 2012 13:33:45 -0400 > Brett Cannon wrote: >> On Mon, Apr 16, 2012 at 13:08, "Martin v. Löwis" wrote: >> So like execute hg diff on the dependent files and if nothing changed then touch the auto-generated file w/ 'touch' to pr

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

Hi, 2012/4/16 Stefan Behnel > > On the down side, we would depend on Cython for evolving . > > Right, although not as a strict dependency. The code would still work just > fine in plain Python. Not quite, we are talking of the imp module here... -- Amaury Forgeot d'Arc __

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

"Martin v. Löwis", 16.04.2012 16:07: >> We have other instances of this (e.g. the Objects/typeslots.inc file >> is generated and checked in), but in the case of importlib, we have >> to use the ./python binary for freezing to avoid bytecode >> incompatibilities, which obviously is a problem if ./py

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Mon, 16 Apr 2012 13:51:35 -0400, Barry Warsaw wrote: > On Apr 16, 2012, at 07:44 PM, Antoine Pitrou wrote: > > >Wouldn't it be better if Python could compile regardless of the > >presence of a hg repository? > > If you want it in your $DISTRO, yes please! My impression is that our usual solu

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Apr 16, 2012, at 07:44 PM, Antoine Pitrou wrote: >Wouldn't it be better if Python could compile regardless of the >presence of a hg repository? If you want it in your $DISTRO, yes please! -Barry ___ Python-Dev mailing list Python-Dev@python.org http

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Mon, 16 Apr 2012 13:33:45 -0400 Brett Cannon wrote: > On Mon, Apr 16, 2012 at 13:08, "Martin v. Löwis" wrote: > > > > So like execute hg diff on the dependent files and if nothing changed > > > then touch the auto-generated file w/ 'touch' to prevent future attempts > > > to execute the targe

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Mon, Apr 16, 2012 at 11:32 AM, Brett Cannon wrote: > > > On Mon, Apr 16, 2012 at 13:04, "Martin v. Löwis" wrote: >> >> > I don't see how depending on Cython is better than depending on having >> > an existing Python.  If the only benefit is semi-readable code, surely >> > we do have source cod

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Mon, Apr 16, 2012 at 13:08, "Martin v. Löwis" wrote: > > So like execute hg diff on the dependent files and if nothing changed > > then touch the auto-generated file w/ 'touch' to prevent future attempts > > to execute the target? > > Exactly. There might be something better than hg diff, perh

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Mon, Apr 16, 2012 at 13:04, "Martin v. Löwis" wrote: > > I don't see how depending on Cython is better than depending on having > > an existing Python. If the only benefit is semi-readable code, surely > > we do have source code for the pre-frozen module, and it is just a matter > > of convin

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

> So like execute hg diff on the dependent files and if nothing changed > then touch the auto-generated file w/ 'touch' to prevent future attempts > to execute the target? Exactly. There might be something better than hg diff, perhaps some form of hg status. Regards, Martin __

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

> I don't see how depending on Cython is better than depending on having > an existing Python. If the only benefit is semi-readable code, surely > we do have source code for the pre-frozen module, and it is just a matter > of convincing hg that the bytecode is binary, not text? Cython-generated C

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Mon, Apr 16, 2012 at 12:31, Antoine Pitrou wrote: > On Mon, 16 Apr 2012 12:15:16 -0400 > "R. David Murray" wrote: > > > > I don't see how depending on Cython is better than depending on having > > an existing Python. If the only benefit is semi-readable code, surely > > we do have source cod

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Mon, 16 Apr 2012 12:15:16 -0400 "R. David Murray" wrote: > > I don't see how depending on Cython is better than depending on having > an existing Python. If the only benefit is semi-readable code, surely > we do have source code for the pre-frozen module, and it is just a matter > of convinci

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Mon, 16 Apr 2012 11:21:34 -0400, Brett Cannon wrote: > On Mon, Apr 16, 2012 at 10:07, "Martin v. Löwis" wrote: > > > > We have other instances of this (e.g. the Objects/typeslots.inc file > > > is generated and checked in), but in the case of importlib, we have > > > to use the ./python bina

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Mon, Apr 16, 2012 at 10:07, "Martin v. Löwis" wrote: > > We have other instances of this (e.g. the Objects/typeslots.inc file > > is generated and checked in), but in the case of importlib, we have > > to use the ./python binary for freezing to avoid bytecode > > incompatibilities, which obvio

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

> We have other instances of this (e.g. the Objects/typeslots.inc file > is generated and checked in), but in the case of importlib, we have > to use the ./python binary for freezing to avoid bytecode > incompatibilities, which obviously is a problem if ./python isn't > built yet. As for dependenc

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

Antoine Pitrou, 16.04.2012 13:13: > On Mon, 16 Apr 2012 09:54:41 +0200 > Stefan Behnel wrote: >> >> The new import cache broke Cython's load of on-the-fly compiled extension >> modules, which naively used "__import__(module_name)" after building them. >> I could fix that by moving to "imp.load_dyna

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On Mon, 16 Apr 2012 09:54:41 +0200 Stefan Behnel wrote: > > The new import cache broke Cython's load of on-the-fly compiled extension > modules, which naively used "__import__(module_name)" after building them. > I could fix that by moving to "imp.load_dynamic()" (we know where we put > the compi

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

Brett Cannon, 14.04.2012 20:12: > My multi-year project -- started in 2006 according to my blog -- to rewrite > import in pure Python and then bootstrap it into CPython as *the* > implementation of __import__() is finally over (mostly)! Hopefully I didn't > break too much code in the process. =) W

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

On 14.04.2012 20:12, Brett Cannon wrote: My multi-year project -- started in 2006 according to my blog -- to rewrite import in pure Python and then bootstrap it into CPython as *the* implementation of __import__() is finally over (mostly)! Hopefully I didn't break too much code in the process. =)

[Python-Dev] importlib is now bootstrapped (and what that means)

My multi-year project -- started in 2006 according to my blog -- to rewrite import in pure Python and then bootstrap it into CPython as *the* implementation of __import__() is finally over (mostly)! Hopefully I didn't break too much code in the process. =) Now this is "mostly" finished because the

Re: [Python-Dev] importlib cleared for merging into default!

On 7 March 2012 23:05, Brett Cannon wrote: > At the language summit today I got clearance to merge my importlib bootstrap > branch (http://hg.python.org/sandbox/bcannon#bootstrap_importlib) thanks to > performance being about 5% slower using the normal_startup (which, as Thomas > Wouters said, is

[Python-Dev] importlib cleared for merging into default!

At the language summit today I got clearance to merge my importlib bootstrap branch (http://hg.python.org/sandbox/bcannon#bootstrap_importlib) thanks to performance being about 5% slower using the normal_startup (which, as Thomas Wouters said, is less than the difference of using the newest gcc in

Re: [Python-Dev] importlib quest

On Mon, Feb 6, 2012 at 14:49, Antoine Pitrou wrote: > On Mon, 6 Feb 2012 09:57:56 -0500 > Brett Cannon wrote: > > Thanks for any help people can provide me on this now 5 year quest to get > > this work finished. > > Do you have any plan to solve the performance issue? > I have not even looked a

Re: [Python-Dev] importlib quest

On Mon, 6 Feb 2012 20:49:48 +0100 Antoine Pitrou wrote: > On Mon, 6 Feb 2012 09:57:56 -0500 > Brett Cannon wrote: > > Thanks for any help people can provide me on this now 5 year quest to get > > this work finished. > > Do you have any plan to solve the performance issue? > > $ ./python -m tim

[Python-Dev] importlib quest

On Mon, 6 Feb 2012 09:57:56 -0500 Brett Cannon wrote: > Thanks for any help people can provide me on this now 5 year quest to get > this work finished. Do you have any plan to solve the performance issue? $ ./python -m timeit -s "import sys; mod='struct'" \ "__import__(mod); del sys.modules[mo

Re: [Python-Dev] importlib

I have updated the benchmark to now measure importing source w/o writing bytecode, importing source & writing bytecode, and importing bytecode w/ source (as I don't care about sourceless import performance). Now, before you look at these numbers, realize that I have not once tried to profile impor

Re: [Python-Dev] importlib

On Wed, 14 Jul 2010 23:06:58 -0700 Brett Cannon wrote: > > > > In any case, here my results under a Linux system: > > > > $ ./python -m importlib.test.benchmark > > sys.modules [ 323782 326183 326667 ] best is 326667 > > Built-in module [ 33600 33693 33610 ] best is 33693 > > > > $ ./python -m imp

Re: [Python-Dev] importlib

anatoly techtonik wrote: What about web-applications? Is that true that for FastCgi or mod_wsgi deamon mode interpreter and application is started only once per say 100 requests? Yes. Only CGI programs reload on every use. FCGI/WSGI programs run more or less forever, and are only loaded on

Re: [Python-Dev] importlib

On Thu, Jul 15, 2010 at 2:55 PM, Nick Coghlan wrote: >>> In any case, here my results under a Linux system: >>> >>> $ ./python -m importlib.test.benchmark >>> sys.modules [ 323782 326183 326667 ] best is 326667 >>> Built-in module [ 33600 33693 33610 ] best is 33693 >>> >>> $ ./python -m importlib

Re: [Python-Dev] importlib

On Thu, Jul 15, 2010 at 4:06 PM, Brett Cannon wrote: >> In any case, here my results under a Linux system: >> >> $ ./python -m importlib.test.benchmark >> sys.modules [ 323782 326183 326667 ] best is 326667 >> Built-in module [ 33600 33693 33610 ] best is 33693 >> >> $ ./python -m importlib.test.b

Re: [Python-Dev] importlib

On Wed, Jul 14, 2010 at 13:01, Antoine Pitrou wrote: > On Wed, 14 Jul 2010 12:33:55 -0700 > Brett Cannon wrote: >> >> So I started writing benchmark code in anticipation of needing to prove a >> minimal performance difference to justify bootstrapping importlib. Right now >> it only compares impor

[Python-Dev] importlib

On Wed, 14 Jul 2010 12:33:55 -0700 Brett Cannon wrote: > > So I started writing benchmark code in anticipation of needing to prove a > minimal performance difference to justify bootstrapping importlib. Right now > it only compares importing from sys.modules and built-in modules. You can > run it