Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-02-02 Thread Dirkjan Ochtman
On Tue, Feb 2, 2010 at 23:54, Collin Winter wrote: > Done. The diff is at > http://codereview.appspot.com/186247/diff2/5014:8003/7002. I listed > Cython, Shedskin and a bunch of other alternatives to pure CPython. > Some of that information is based on conversations I've had with the > respective

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-02-02 Thread Ben Finney
Barry Warsaw writes: > If you have to manually blow away a particular pyc file, > folder-per-folder makes it much easier to find exactly what you want > to blow away without have to search up the file system, and then back > down again to find the pyc file to delete. How many ..'s does it take >

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-02-02 Thread Ben Finney
Barry Warsaw writes: > I suppose this is going to be very subjective, but in skimming the > thread it seems like most people like putting the byte code cache > artifacts in subdirectories (be they siblings or folder-per-folder). I don't understand the distinction you're making between those two

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-02-02 Thread Barry Warsaw
On Jan 31, 2010, at 09:30 PM, Martin v. Löwis wrote: >If a single pyc folder is used, I think an additional __source__ >attribute would be needed to indicate what source file time stamp had >been checked (if any) to determine that the byte code file is current. This is a good point. __file__ is

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-02-02 Thread Barry Warsaw
On Jan 31, 2010, at 12:36 PM, Georg Brandl wrote: >Not really -- much of the code I've seen that tries to guess the source >file name from a __file__ value just does something like this: > > if fname.lower().endswith(('.pyc', '.pyo')): fname = fname[:-1] > >That's not compatible with using .pyr,

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-02-02 Thread Barry Warsaw
On Jan 31, 2010, at 03:07 PM, Ben Finney wrote: >In other words, my understanding is that the current PEP would have the >following tree for an example project:: > >foo/ >__init__.py >__init__.pyr/ >deadbeef.pyc >decafbad.pyc >lorem.py >l

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-02-02 Thread Barry Warsaw
On Jan 31, 2010, at 01:44 PM, Nick Coghlan wrote: >We deliberate don't document -U because its typical effect is "break the >world" - it makes all strings unicode in 2.x. As an aside, I think this should be documented *somewhere* other than just in import.c! I'd totally forgotten about it until

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-02-02 Thread Barry Warsaw
On Jan 30, 2010, at 11:21 PM, Vitor Bosshard wrote: >Why not: > >foo.py >foo.pyc # < 2.7 or < 3.2 >foo.27.pyc >foo.32.pyc >etc. Because this clutters the module's directory more than it does today, which I considered to be a negative factor. And as others have pointed out, there isn't a one-to-o

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-02-02 Thread Barry Warsaw
I have to say up front that I'm somewhat shocked at how quickly this thread has exploded! Since I'm sprinting this week, I haven't thoroughly read every message and won't have time tonight to answer every question, but I'll try to pick out some common ideas. I really appreciate everyone's input a

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-02-02 Thread Bob Ippolito
On Sun, Jan 31, 2010 at 11:16 AM, Guido van Rossum wrote: > Whoa. This thread already exploded. I'm picking this message to > respond to because it reflects my own view after reading the PEP. > > On Sun, Jan 31, 2010 at 4:13 AM, Hanno Schlichting wrote: >> On Sun, Jan 31, 2010 at 1:03 PM, Simon C

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-02-02 Thread Guido van Rossum
On Tue, Feb 2, 2010 at 5:41 PM, Glenn Linderman wrote: > On approximately 2/2/2010 4:28 PM, came the following characters from the > keyboard of Guido van Rossum: >> >> Argh. zipfiles are way to complex to be writing. > > Agreed.  But in reading that, it somehow triggered a question: does > zipimp

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-02-02 Thread Reid Kleckner
On Tue, Feb 2, 2010 at 8:57 PM, Collin Winter wrote: >> Wouldn't it be possible to have the compiler approach work >> in three phases in order to reduce the memory footprint and >> startup time hit, ie. >> >>  1. run an instrumented Python interpreter to collect all >>    the needed compiler infor

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-02-02 Thread Collin Winter
Hey MA, On Fri, Jan 29, 2010 at 11:14 AM, M.-A. Lemburg wrote: > Collin Winter wrote: >> I added startup benchmarks for Mercurial and Bazaar yesterday >> (http://code.google.com/p/unladen-swallow/source/detail?r=1019) so we >> can use them as more macro-ish benchmarks, rather than merely starting

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-02-02 Thread Glenn Linderman
On approximately 2/2/2010 4:28 PM, came the following characters from the keyboard of Guido van Rossum: Argh. zipfiles are way to complex to be writing. Agreed. But in reading that, it somehow triggered a question: does zipimport only work for zipfiles, or does it work for any archive format

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-02-02 Thread Craig Citro
> Done. The diff is at > http://codereview.appspot.com/186247/diff2/5014:8003/7002. I listed > Cython, Shedskin and a bunch of other alternatives to pure CPython. > Some of that information is based on conversations I've had with the > respective developers, and I'd appreciate corrections if I'm ou

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-02-02 Thread Guido van Rossum
Argh. zipfiles are way to complex to be writing. If you want to use zipfiles, compile your whole world ahead of time, stuff it into a zipfile, and install / distribute that. But for the automatic writing of bytecode files as a side effect of importing the source code, please let the filesystem do i

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-02-02 Thread Neil Schemenauer
Nick Coghlan wrote: > Henning von Bargen wrote: >> The solution is so obvious: >> >> Why not use a .pyr file that is internally a zip file? I think a Zip file might be the right approach too. Either you could have directories in the zip file for each version, e.g. 2.7/foo.pyc 3.3/foo.p

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-02-02 Thread Sebastian Rittau
On Sun, Jan 31, 2010 at 12:44:33PM +0100, Georg Brandl wrote: > At least to me, this does not explain why an "unwanted" (why unwanted? If > it's unwanted, set PYTHONDONTWRITEBYTECODE=1) directory is worse than an > "unwanted" file. A directory "feels" different than. For example, typing "ls" in m

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-02-02 Thread Collin Winter
Hey Dirkjan, [Circling back to this part of the thread] On Thu, Jan 21, 2010 at 1:37 PM, Dirkjan Ochtman wrote: > On Thu, Jan 21, 2010 at 21:14, Collin Winter wrote: [snip] >> My quick take on Cython and Shedskin is that they are >> useful-but-limited workarounds for CPython's historically-poor

Re: [Python-Dev] Python 2.6.5

2010-02-02 Thread Raymond Hettinger
+1 On Feb 2, 2010, at 10:08 AM, Barry Warsaw wrote: > I'm thinking about doing a Python 2.6.5 release soon. I've added the > following dates to the Python release schedule Google calendar: > > 2009-03-01 Python 2.6.5 rc 1 > 2009-03-15 Python 2.6.5 final > > This allows us to spend some time on

Re: [Python-Dev] Absolute imports in Python 2.x?

2010-02-02 Thread Brett Cannon
On Tue, Feb 2, 2010 at 05:20, Eric Smith wrote: > Mark Dickinson wrote: >> >> What are the current plans for PEP 328 (the absolute imports PEP) in >> Python 2.x? >> >> The PEP says: >> >> """In Python 2.6, any import statement that results in an >> intra-package import will raise DeprecationWarnin

[Python-Dev] API for VM branches (was: PEP 3146)

2010-02-02 Thread Collin Winter
[Moving to python-ideas; python-dev to bcc] On Tue, Feb 2, 2010 at 2:02 AM, M.-A. Lemburg wrote: > Collin Winter wrote: [snip] >> If such a restrictive plugin-based scheme had been available when we >> began Unladen Swallow, I do not doubt that we would have ignored it >> entirely. I do not like

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-02 Thread Martin v. Löwis
> Although I would be in favor of an atfork callback registration system > (similar to atexit), it seems there is no way to solve the fork() > problem automatically with this. Any attempt to acquire/release locks > automatically will lead to deadlocks, as it is necessary to know the > exact program

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-02 Thread Martin v. Löwis
> Le lundi 01 février 2010 à 23:58 +0100, "Martin v. Löwis" a écrit : >> Interestingly, the POSIX pthread_atfork documentation defines how you >> are supposed to do that: create an atfork handler set, and acquire all >> mutexes in the prepare handler. Then fork, and have the parent and child >> han

[Python-Dev] Python 2.6.5

2010-02-02 Thread Barry Warsaw
I'm thinking about doing a Python 2.6.5 release soon. I've added the following dates to the Python release schedule Google calendar: 2009-03-01 Python 2.6.5 rc 1 2009-03-15 Python 2.6.5 final This allows us to spend some time on 2.6.5 at Pycon if we want. Please let me know if you have any conc

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-02 Thread Nir Aides
Seems the problem under discussion is already taken care of in Python. Possibly remains to verify that the logic described below does not possibly generate deadlocks. >From the Python docs: http://docs.python.org/c-api/init.html "Another important thing to note about threads is their behaviour in

Re: [Python-Dev] Mecurial (was: PEP 3146: Merge Unladen Swallow into CPython)

2010-02-02 Thread M.-A. Lemburg
Nick Coghlan wrote: > M.-A. Lemburg wrote: >> BTW: I also doubt that Mercurial will make any of this easier. >> It makes creating branches easier for non-committers, but the >> problem of having to maintain the branches remains. > > It greatly simplifies the process of syncing the branch with the

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-02 Thread Jesse Noller
On Tue, Feb 2, 2010 at 10:34 AM, Pascal Chambon wrote: > > > > The word "dogma" is a good one in this context however. "We" ( ;-)) have > accepted and promoted the dogma that multiprocessing is the solution to > parallelism in the face of the GIL. While it needn't be applicable in any > and > ever

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-02 Thread Pascal Chambon
The word "dogma" is a good one in this context however. "We" ( ;-)) have accepted and promoted the dogma that multiprocessing is the solution to parallelism in the face of the GIL. While it needn't be applicable in any and every situation, we should make it so that it is applicable often enou

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-02 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 s...@pobox.com wrote: > Tres> Some applications may seem to work when violating this rule, but > Tres> their developers are doomed to hair loss over time. > > Then for us bald guys it should be okay, right? ;-) Sure: you might even grow hair

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-02 Thread skip
Tres> Some applications may seem to work when violating this rule, but Tres> their developers are doomed to hair loss over time. Then for us bald guys it should be okay, right? ;-) Skip ___ Python-Dev mailing list Python-Dev@python.org http://m

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-02-02 Thread Larry Hastings
On Sun, Jan 31, 2010 at 1:03 PM, Simon Cross wrote: I don't know whether I in favour of using a single pyr folder or not but if a single folder is used I'd definitely prefer the folder to be called __pyr__ rather than .pyr. Guido van Rossum wrote: Exactly what I would prefer. I worry that ha

Re: [Python-Dev] Reworking the GIL

2010-02-02 Thread Stephen J. Turnbull
Antoine Pitrou writes: > Yes, I hindsight I think Guido was right. Guido does too. One of the benefits of having a time machine is getting to turn your hindsight into foresight. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org

Re: [Python-Dev] Absolute imports in Python 2.x?

2010-02-02 Thread Eric Smith
Mark Dickinson wrote: What are the current plans for PEP 328 (the absolute imports PEP) in Python 2.x? The PEP says: """In Python 2.6, any import statement that results in an intra-package import will raise DeprecationWarning (this also applies to from <> import that fails to use the relative i

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-02 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Antoine Pitrou wrote: > Tres Seaver palladion.com> writes: >> Note that the "we" in your sentence is not anything like the "quod >> semper quod ubique quod ab omnibus" criterion for accepting dogma: >> mutliprocessing is a tool, and needs to be used a

Re: [Python-Dev] Reworking the GIL

2010-02-02 Thread Larry Hastings
Antoine Pitrou wrote: Also, the MSDN doc (*) says timeBeginPeriod() can have a detrimental effect on system performance; I don't know how much of it is true. (*) http://msdn.microsoft.com/en-us/library/dd757624(VS.85).aspx Indeed it does. This is ancient, dusty wisdom, from the days of 50mhz

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-02 Thread Antoine Pitrou
Tres Seaver palladion.com> writes: > > Note that the "we" in your sentence is not anything like the "quod > semper quod ubique quod ab omnibus" criterion for accepting dogma: > mutliprocessing is a tool, and needs to be used according to its nature, > just as with threading. I don't remember eno

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-02 Thread Antoine Pitrou
pobox.com> writes: > > I don't understand. On Unix-y systems isn't spawn* layered on top of > fork/exec? The point is that exec() relinquishes all the existing resources, so the initial fork() becomes an implementation detail (IIUC). > If you were going to > combine both threading and multipro

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-02 Thread skip
>> I guess spawnl semantic (i.e, like win32's CreateProcess()) can't >> become the default multiprocessing behaviour... Nick> It would also make it much easier to write cross-platform Nick> multiprocessing code (by always using the non-forking semantics Nick> even on fork-capa

Re: [Python-Dev] Reworking the GIL

2010-02-02 Thread Antoine Pitrou
Hello Eric, > I notice that Guido vetoed this idea, but just in case it should come up > again, I have some thoughts that likely have already occurred to people, > but I didn't notice on the list. Yes, I hindsight I think Guido was right. (no, I'm not trying to make myself agreeable :-)) > Also

[Python-Dev] Absolute imports in Python 2.x?

2010-02-02 Thread Mark Dickinson
What are the current plans for PEP 328 (the absolute imports PEP) in Python 2.x? The PEP says: """In Python 2.6, any import statement that results in an intra-package import will raise DeprecationWarning (this also applies to from <> import that fails to use the relative import syntax). In Python

Re: [Python-Dev] Mail Archives

2010-02-02 Thread Nick Coghlan
Chris Rebert wrote: > On Tue, Feb 2, 2010 at 1:50 AM, wrote: >> Hi, >> >> Currently the Python-Dev archives at >> http://mail.python.org/pipermail/python-dev/ doesn't appear to have been >> updated for the last week or so. Is that a known issue? > > I think so: > http://mail.python.org/pipermai

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-02 Thread Nick Coghlan
Pascal Chambon wrote: > I guess spawnl semantic (i.e, like win32's CreateProcess()) can't become > the default multiprocessing behaviour, as too many programs implicitly > rely on the whole sharing of data under unix (and py3k itself is maybe > becoming a little too mature for new compatility break

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-02-02 Thread Nick Coghlan
M.-A. Lemburg wrote: > BTW: I also doubt that Mercurial will make any of this easier. > It makes creating branches easier for non-committers, but the > problem of having to maintain the branches remains. It greatly simplifies the process of syncing the branch with the main line of development so y

Re: [Python-Dev] Reworking the GIL

2010-02-02 Thread Neil Hodgson
Eric Hopper: > I don't suppose it will ever be ported back to Python 2.x?  It doesn't > look like the whole GIL concept has changed much between Python 2.x and > 3.x so I expect back-porting it would be pretty easy. There was a patch but it has been rejected. http://bugs.python.org/issue7753

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-02 Thread Pascal Chambon
Although I would be in favor of an atfork callback registration system (similar to atexit), it seems there is no way to solve the fork() problem automatically with this. Any attempt to acquire/release locks automatically will lead to deadlocks, as it is necessary to know the exact program wor

Re: [Python-Dev] Mail Archives

2010-02-02 Thread Chris Rebert
On Tue, Feb 2, 2010 at 1:50 AM, wrote: > Hi, > > Currently the Python-Dev archives at > http://mail.python.org/pipermail/python-dev/ doesn't appear to have been > updated for the last week or so. Is that a known issue? I think so: http://mail.python.org/pipermail/python-dev/2010-January/097388.h

[Python-Dev] Mail Archives

2010-02-02 Thread Ben.Young
Hi, Currently the Python-Dev archives at http://mail.python.org/pipermail/python-dev/ doesn't appear to have been updated for the last week or so. Is that a known issue? Thanks, Ben Ben Young - Senior Software Engineer SunGard - Enterprise House, Vision Park, Histon, Cambridge, CB24 9ZR Tel +44

Re: [Python-Dev] Reworking the GIL

2010-02-02 Thread Eric Hopper
I recently saw the video of David Beazley's presentation on how poorly the old GIL implementation handled certain cases and thought "I can fix that!". Unfortunately for me, someone else has beaten me to it, and done a somewhat better job than I would've because I wasn't thinking of doing anything

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-02-02 Thread M.-A. Lemburg
Collin Winter wrote: > On Mon, Feb 1, 2010 at 11:17 AM, M.-A. Lemburg wrote: >> Collin Winter wrote: >>> I think this idea underestimates a) how deeply the current CPython VM >>> is intertwined with the rest of the implementation, and b) the nature >>> of the changes required by these separate VMs