[Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows
Hi all, During the huge thread about PEP 394, a suggestion was made that a launcher for Python on Windows could be implemented which would allow for some of the guidelines in that PEP to apply to the Windows version of Python. I've attached the first draft of a PEP for such a launcher and put the first draft of the reference implementation (implemented in Python 3) at http://starship.python.net/crew/mhammond/downloads/python_launcher.py The reference implementation is fairly complete WRT the features described in the PEP. However, I'm sure there will be comments on both the PEP itself and the implementation, so I'm putting them up for discussion now rather than spending more time polishing things which may end up changing. I'll start work on a C implementation once there is broad agreement on the functionality in the reference implementation. All comments welcome, including, but not limited to: * Is this really PEP material, or will turning the PEP into a regular spec be suitable? * If it is a PEP, is it Standards Track or Informational? * Does the functionality described strike the right balance between simplicity and usefulness? * Does the strategy have general support from Martin, who as the person making Python distributions would need to be involved in at least some aspects of it (specifically, having the installer MSI install the launcher and update the file associations to reference it). Thanks, Mark PEP: ?? Title: Python launcher for Windows Version: $Revision$ Last-Modified: $Date$ Author: Mark Hammond mhamm...@skippinet.com.au Status: Draft Type: Standards Track or Informational ? Content-Type: text/x-rst Created: 15-Mar-2011 Abstract This PEP describes a Python launcher for the Windows platform. A Python launcher is a single executable which uses a number of heuristics to locate a Python executable and launch it with a specified command line. Rationale Unix-like operating systems (referred to simply as Unix in this PEP) allow scripts to be executed as if they were executable images by examining the script for a shebang line which specified the actual executable to be used to run the script. This is described in detail in the evecve(2) man page [1]. Additionally, these operating systems provide symbolic-links to Python executables in well-known directories. For example, many systems will have a link /usr/bin/python which references a particular version of Python installed under the operating-system. These symbolic links allow Python to be executed without regard for where Python it actually installed on the machine (eg, without requiring the path where Python is actually installed to be referenced in the shebang line or in the PATH. PEP 394 'The python command on Unix-Like Systems' [2] describes additional conventions for more fine-grained specification of a particular Python version. These 2 facilities combined allow for a portable and somewhat predictable way of both starting Python interactively and for allowing Python scripts to execute. This PEP describes an implementation of a launcher which can offer the same benefits for Python on the Windows platform. An overview of the launcher. This PEP outlines the general functionality and key guidelines of a launcher for Windows. It is accompianied by reference implemention, written in Python, which defines the detailed implementation. Over time, changes to the implementation may be desired - if the changes adhere to the guidelines in this PEP and have been made following the standard Python development model, this PEP need not change. In other words, this PEP makes no attempt to describe in detail every feature offered by the launcher, but instead to offer guidelines the launcher should adhere to. The actual implementation of the launcher will be written in C and will not link directly or indirectly with Python, but the Python based reference implementation should always remain the canonical description of the functionality. Any changes proposed to the launcher functionality should first be made and agreed to in the Python implementation, after which time the C implementation can be changed to reflect the new reference. Any deviations between the functionality of the reference implementation versus the C implementation due should imply the C implementation is in error. It is expected that 2 versions of the launcher will be needed - one which is a console program and one which is a windows (ie, GUI) program. These 2 launchers correspond to the 'python.exe' and 'pythonw.exe' executables which currently ship with Python. The console launcher will be named 'py.exe' and the Windows one named 'pyw.exe'. The windows (ie, GUI) version of the launcher will attempt to locate and launch
Re: [Python-Dev] API deprecations in Python 3, from a Python 2 perspective
On Sat, Mar 19, 2011 at 03:42, Nick Coghlan ncogh...@gmail.com wrote: 5. The parallel evolution of the 2.x and 3.x line meant that the first version of 2.x with the relevant warning was released only ~7 months or so before the version of 3.2 where the API was removed An additional issue that makes it particularly problematic for me is that this is a part of moving a whole framework, the Zope Component Architecture And Friends (aka the Bicycle toolkit) that contains a lot of hard to port projects like zope.testing and zc.buildout. This means that it's so far take three years of porting (starting with efforts to port Setuptools in 2008) and isn't done yet. It's extremely frustrating that in the middle of this porting effort the target changes, so things I *thought* was done and ported, are now suddenly not done anymore. It becomes a sisyphean task to port to Python 3, and I don't fancy myself ending up in such a boring part of hell. :-) If I had instead ignored Python 3 completely until this year, then I this wouldn't have been a problem. As a result it feels like early porters like me just got our fingers whacked. But it all comes down to what Guido mentioned in his fie-side chat (at least how I understood him): The pace of change for Python needs to slow down a bit, at least until the migration to Python 3 both for Python implementations and general use has been completed. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hg: inter-branch workflow
On Sat, Mar 19, 2011 at 12:22 PM, Jesus Cea j...@jcea.es wrote: On 19/03/11 03:14, Nick Coghlan wrote: (i.e. start moving towards more of a style of development where code doesn't land in the main repository until it has been vetted by the buildbots *first*). Unfortunately, this is *not* good enough. Basically what you are talking about is equivalent to automatic rebasing (although without history loss). The rebased code *must* be retested in the new context anyway. (Linus famously flamed DaveM over this issue on lkml. Sorry, I don't have the URL.) To the extent the buildbots are not overloaded, this strategy will indeed save developer time, as most changes are more or less independent of each other (that's why automated merging works at all well), and most of the time something that passes in a branch will also pass after merging. So it's probably easier, and certainly less wearing on other developers, if you detect breakage in the branch rather than waiting for it to happen in the trunk. However, such early detection is not guaranteed because not all semantic conflicts are syntactic conflicts. Ie, the merge may succeed but the code break. In fact I am thinking about being able to push a patch to the buildbots and, if the patch applies cleanly to the branch tip, actually apply it, build test. This is what Bazaar does with its PQM service (patch queue manager). I don't see why that wouldn't be usable with Mercurial, although if it uses Bazaar internal APIs (a lot of Bazaar-related projects do) it might be mildly tedious to port). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Module version variable
On Fri, 18 Mar 2011 20:12:19 -0700 Toshio Kuratomi a.bad...@gmail.com wrote: There is a section in PEP8 about __version__ but it serves a slightly different purpose there: Version Bookkeeping If you have to have Subversion, CVS, or RCS crud in your source file, do it as follows. __version__ = $Revision: 88433 $ # $Source$ This should be updated (or rather, removed) now that we use Mercurial... Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.2): Issue 7391: Remove questionable and outdated HOWTO document with permission
On Sat, 19 Mar 2011 02:22:42 +0100 raymond.hettinger python-check...@python.org wrote: http://hg.python.org/cpython/rev/80ff78425419 changeset: 68671:80ff78425419 branch: 3.2 parent: 68665:f832ca852329 user:Raymond Hettinger pyt...@rcn.com date:Fri Mar 18 18:22:28 2011 -0700 summary: Issue 7391: Remove questionable and outdated HOWTO document with permission from its author. How is this questionable? I found it quite useful when trying to warn beginners about anti-idioms. Is there another resource which replaces it? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Module version variable
I do distinctly recall __version__ being standardized for other purposes too, but I have no idea how to find that reference... It probably was well before 2000. Maybe you were thinking about Pydoc, which will display __version__ in a dedicated section of the doc. Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
I have a trivial little documentation patch for csv.rst. I committed it locally, then I pulled and merged: cpython% hg pull pulling from ssh://h...@hg.python.org/cpython searching for changes adding changesets adding manifests adding file changes added 94 changesets with 422 changes to 154 files (+1 heads) (run 'hg heads' to see heads, 'hg merge' to merge) cpython% hg merge 132 files updated, 0 files merged, 0 files removed, 0 files unresolved (branch merge, don't forget to commit) hg heads showed my changeset: changeset: 68585:c63d7374b89a user:Skip Montanaro s...@pobox.com date:Sat Mar 19 09:09:30 2011 -0500 summary: Mention RFC 4180. Based on input by Tony Wallace in issue 11456. I committed. Now: changeset: 68680:64eeb4cd4b56 tag: tip parent: 68585:c63d7374b89a parent: 68679:dfceb98767c0 user:Skip Montanaro s...@pobox.com date:Sat Mar 19 09:15:28 2011 -0500 summary: commit merge The dev guide says something about collapsing changesets. Is that collapsing commits within a changeset or collapsing multiple changesets (whatever that might be)? Do I need this for a trivial change? Can I just push at this point? Once pushed, how does it get merged into the main codebase? Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
hg heads showed my changeset: changeset: 68585:c63d7374b89a user:Skip Montanaro s...@pobox.com date:Sat Mar 19 09:09:30 2011 -0500 summary: Mention RFC 4180. Based on input by Tony Wallace in issue 11456. I committed. Now: changeset: 68680:64eeb4cd4b56 tag: tip parent: 68585:c63d7374b89a parent: 68679:dfceb98767c0 user:Skip Montanaro s...@pobox.com date:Sat Mar 19 09:15:28 2011 -0500 summary: commit merge The dev guide says something about collapsing changesets. Is that collapsing commits within a changeset or collapsing multiple changesets (whatever that might be)? It's only the latter. If you had a number of commits done just to implement the feature, some with commit messages such as fix typo, revert bogus change, more cleanup, etc., people don't want to see this in Python's history, so you would have to come up with a single patch for the entire feature and commit that separately. Do I need this for a trivial change? No. The essential change is in a single changeset (c63d7374b89a), which is how it should be. Can I just push at this point? Yes. If somebody pushed something else meanwhile, you'll have to merge again. Once pushed, how does it get merged into the main codebase? Pushing the change will exactly do that. Your merge (64eeb4cd4b56) has your change (c63d7374b89a) as it's parent, so 64eeb4cd4b56 now contains both the current state of cpython plus your change. If somebody else now makes other changes that succeeded your merge, you will need to merge them again, so that, at the time of your push, your most recent merge is newer than cpython's default head. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.2): Issue 7391: Remove questionable and outdated HOWTO document with permission
On Sat, 19 Mar 2011 13:57:02 +0100, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 19 Mar 2011 02:22:42 +0100 raymond.hettinger python-check...@python.org wrote: http://hg.python.org/cpython/rev/80ff78425419 changeset: 68671:80ff78425419 branch: 3.2 parent: 68665:f832ca852329 user:Raymond Hettinger pyt...@rcn.com date:Fri Mar 18 18:22:28 2011 -0700 summary: Issue 7391: Remove questionable and outdated HOWTO document with permission from its author. How is this questionable? I found it quite useful when trying to warn beginners about anti-idioms. Is there another resource which replaces it? Indeed. I spent some time a few months back updating a couple sections of that howto. Perhaps the better thing to do would be to update it instead of deleting it (although I admit I haven't read through the whole thingan issue was submitted for a particular section and I updated that section and the other one talking about the same topic (sorry I don't remember which one off the top of my head)). -- R. David Murray http://www.bitdance.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
On Sat, 19 Mar 2011 09:25:07 -0500, s...@pobox.com wrote: I have a trivial little documentation patch for csv.rst. I committed it locally, then I pulled and merged: Note that if you want doc changes to appear in all the current doc sets (2.7, 3.2, dev), then you should start with patching 3.2 and merge it into default, and also patch 2.7, just like any other bug-fix commit. Making doc changes to all current doc sets is our normal practice (unless, of course, they are about new features). You can also optionally start your patching with 3.1, since the 3.1 docs will be re-generated one last time for its upcoming last bugfix release. It's easy enough (now that we no longer have to deal with svnmerge) that I've been doing that, myself. -- R. David Murray http://www.bitdance.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows
Not all of the ideas below are complementary to each other, some are either or, to allow different thoughts to be inspired or different directions to be taken. Thanks for starting a PEP. On 3/18/2011 11:02 PM, Mark Hammond wrote: The launcher should be as simple as possible (but no simpler.) The launcher could be simpler if it isn't used for launching interactive interpreters as well as script references via Windows associations (more about that after the next quote). The launcher could be simpler if it just read through the file of its first parameter until it finds a line starting with #@ (process as a Windows version of Unix #!) or starting without a # (error case). This avoids the need parse such lines. So in reality, you are not trying to make launcher as simple as possible, but rather attempting to make some perceived usage case simple, and make the launcher more complex (and harder to define, document, and implement) rather than suggesting that since Unix and Windows are different, that maybe they should have different lines in the script to help launch them. Remember, the typical Windows user is not likely to place a #! line in their scripts in the first place, so teaching them what a Unix #! line is, and how the parameter after it should be something that Windows doesn't even use, and the launcher has to work hard to interpret, is not as simple as possible. The launcher could be simpler if the Python installer placed versioned Python executables on the PATH. Unfortunately, historically it hasn't. If it did, would, or the launcher installer would place them there for pre-existing versions of Python, then the launcher could work by launching the appropriate version of Python, expecting Windows to find it on the PATH. The PEP doesn't address the level of internal complexity of the launcher necessary to find where Python has been installed, neither for CPython, nor for the alternate implementations to be supported. The launcher could be simpler if a directory \usr\bin were created under Windows Program Files, placed on the PATH, and %ProgramFiles% prepended to the Unix #! line, with the Python/Jython/Cython installers placing appropriately versioned executables in that directory. Could even start a trend for programs ported from Unix. One could even place an env program there, for more simplicity. * When used to launch an interactive Python interpreter, the launcher will support the first command-line argument optionally be a version specifier in the form -n[.n] (where n is a single integer) to nominate a specific version be used. For example, while py.exe may locate and launch the latest Python 2.x implementation installed, a command-line such as py.exe -3 could specify the latest Python 3.x implementation be launched, while py.exe -2.6 could specify Python 2.6 be located and launched. If a Python 2.x implementation is desired to be launched with the -3 flag, the command-line would need to be similar to py.exe -2 -3 (or the specific version of Python could obviously be launched manually without use of this launcher.) I think that a python launcher that is on the PATH that could be used to launch an interactive Python, should be different than one that is used to launch .py[w] scripts. The above paragraph raises the issues below, which I think are confusing enough to justify this, IMO. Certainly, if the same python launcher is used for both cases, a lot more clarity around parameter handling must be provided. 1) python should be invoked interactively by typing python or pythonX[.Y] at the CMD prompt, not py. This can be done without a launcher, if appropriate versioned pythons are placed on the PATH. The launcher is really and only needed for handling .py[w] scripts, which, in the Windows way of thinking, can only be associated with one specific, system-wide configured version of Python (presently, the latest one wins). The script itself is not examined to modify such an association. The Unix !# line provides such modification on Unix. 2) If the launcher provides command line options for the benefit of launching interactive Python interpreters, those command line options can have data puns with script names, or can conflict with actual Python options. I believe Python 2 already has a -3 option, for example. And Windows users are not trained that - introduces an option syntax, but rather /. Most _programmer_ users would probably be aware of - as an option syntax, but Python is used as a language for non-programmers in some circles, and few Windows non-programmers understand / much less - and not even command lines very well. So not using a launcher for launching interactive Python sidesteps all that: Python itself is introduced and taught, and no need to teach about (or even have) launcher options that could
Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows
On Sat, Mar 19, 2011 at 4:44 AM, Glenn Linderman v+pyt...@g.nevcal.com wrote: The launcher could be simpler if the Python installer placed versioned Python executables on the PATH. Unfortunately, historically it hasn't. If it did, would, or the launcher installer would place them there for pre-existing versions of Python, then the launcher could work by launching the appropriate version of Python, expecting Windows to find it on the PATH. The PEP doesn't address the level of internal complexity of the launcher necessary to find where Python has been installed, neither for CPython, nor for the alternate implementations to be supported. The complexity isnt all that much since python has installed Registry keys in known locations for a long time, it could/would just use these to locate the proper python version The launcher could be simpler if a directory \usr\bin were created under Windows Program Files, placed on the PATH, and %ProgramFiles% prepended to the Unix #! line, with the Python/Jython/Cython installers placing appropriately versioned executables in that directory. Could even start a trend for programs ported from Unix. One could even place an env program there, for more simplicity. This is not about trying to get windows to behave more like *nix in general it is about allowing python to have interoperability between windows and unix with well known shebang lines I think that a python launcher that is on the PATH that could be used to launch an interactive Python, should be different than one that is used to launch .py[w] scripts. The above paragraph raises the issues below, which I think are confusing enough to justify this, IMO. Certainly, if the same python launcher is used for both cases, a lot more clarity around parameter handling must be provided. The pep specifically stated that there would be py.exe and pyw.exe 1) python should be invoked interactively by typing python or pythonX[.Y] at the CMD prompt, not py. This can be done without a launcher, if appropriate versioned pythons are placed on the PATH. The launcher is really and only needed for handling .py[w] scripts, which, in the Windows way of thinking, can only be associated with one specific, system-wide configured version of Python (presently, the latest one wins). The script itself is not examined to modify such an association. The Unix !# line provides such modification on Unix. Adding each version of python to the PATH is not going to happen on windows the devs have always been resistant to this (A little annoying yes, but it is what it is) 2) If the launcher provides command line options for the benefit of launching interactive Python interpreters, those command line options can have data puns with script names, or can conflict with actual Python options. I believe Python 2 already has a -3 option, for example. And Windows users are not trained that - introduces an option syntax, but rather /. Most _programmer_ users would probably be aware of - as an option syntax, but Python is used as a language for non-programmers in some circles, and few Windows non-programmers understand / much less - and not even command lines very well. So not using a launcher for launching interactive Python sidesteps all that: Python itself is introduced and taught, and no need to teach about (or even have) launcher options that could possibly conflict and confuse, in addition to Python options that may conflict with script names already. (I have seen lots of Windows users use leading punctuation characters on filenames to affect sort order and grouping of files in My Documents, not knowing they can create subdirectories/subfolders, or not wanting to bother with them, since all their applications default to opening things from My Documents.) The command line options I disagree with as well. If the user wants to test a script that has a shebang of python2 with python3 then they should explicitly launch it that way just like you would have to do on *nix 3) Unix !# lines can have embedded options after the program name on the line. Such options would be another source of potential conflict with launcher options, if the launcher has options for use with launching interactive interpreters. Item 3 is also an issue for the PEP even apart from its use as an interactive Python launcher; since options may exist on the Unix #! line, a discussion of how and if they are handled by the launcher should be included in the PEP. They should be passed exactly as they are on the shebang line * Environment varialbes will be used to override the semantics for determining exactly what version of Python will be used. For example, while a shebang line of /usr/bin/python2 will automatically locate a Python 2.x implementation, an environment variable can override exactly which Python 2.x implementation will be chosen. Similarly for
Re: [Python-Dev] cpython (3.2): Issue 7391: Remove questionable and outdated HOWTO document with permission
On 19.03.2011 16:30, R. David Murray wrote: On Sat, 19 Mar 2011 13:57:02 +0100, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 19 Mar 2011 02:22:42 +0100 raymond.hettinger python-check...@python.org wrote: http://hg.python.org/cpython/rev/80ff78425419 changeset: 68671:80ff78425419 branch: 3.2 parent: 68665:f832ca852329 user:Raymond Hettinger pyt...@rcn.com date:Fri Mar 18 18:22:28 2011 -0700 summary: Issue 7391: Remove questionable and outdated HOWTO document with permission from its author. How is this questionable? I found it quite useful when trying to warn beginners about anti-idioms. Is there another resource which replaces it? Indeed. I spent some time a few months back updating a couple sections of that howto. Perhaps the better thing to do would be to update it instead of deleting it (although I admit I haven't read through the whole thingan issue was submitted for a particular section and I updated that section and the other one talking about the same topic (sorry I don't remember which one off the top of my head)). +1. (Also I don't understand why we'd need permission from an author to *remove* content.) Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
On Sat, 19 Mar 2011 09:25:07 -0500 s...@pobox.com wrote: I have a trivial little documentation patch for csv.rst. I committed it locally, then I pulled and merged: cpython% hg pull pulling from ssh://h...@hg.python.org/cpython searching for changes adding changesets adding manifests adding file changes added 94 changesets with 422 changes to 154 files (+1 heads) 94 changesets? If you want to avoid risking conflicts, you should hg pull and hg up (or hg pull -u) before you start working on something (just like you svn up'ed before working on something). The dev guide says something about collapsing changesets. Is that collapsing commits within a changeset or collapsing multiple changesets (whatever that might be)? Do I need this for a trivial change? Can I just push at this point? Once pushed, how does it get merged into the main codebase? Sincerely, I would really recommend that you read a Mercurial tutorial. We could answer all your questions one by one but that wouldn't help you much if you don't understand the concepts. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] buildbot VM exhausted in test_crashers
I've run out of virtual memory on my Ubuntu buildbot today (and I think the FreeBSD buildbots a few days ago). Those have physically OOMd/restarted the VM. The Windows buildbots have been giving me VM warnings, but appear to have gotten through it. I believe it's the test_crashers test, which when I observed the Ubuntu buildbot when trying to bring it back online suddenly spikes to over 500MB allocated to the Python process (which is the limit of physical memory on that box), then causes an overall OOM and crash. The problem is that the failing build is automatically retried when I try to reconnect the buildbot, so it just get rights into trouble again. The comment in that build's changeset looks like it might be getting rolled back? Assuming that's the case, is there a way to flush any pending builds for my slaves before I bring them back online? Thanks. -- David PS: Neither FreeBSD nor my Ubuntu slave are actually up though the master thinks they are. The pinging option worked once for the Ubuntu slave but at the moment I can't seem to get the master to think that any of them are really disconnected. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows
* Is this really PEP material, or will turning the PEP into a regular spec be suitable? It's PEP material if it is contentious, which I believe it is. I, for example, will find issues with it if the implementation uses CreateProcess at some point - the launcher should IMO run Python directly, rather than creating a new process. This, in turn, will probably mean that you cannot actually implement the PEP in Python. Others may find issues if the PEP proposes to change the standard association for .py (and also issues if it doesn't). * If it is a PEP, is it Standards Track or Informational? If you are proposing to change the CPython code base (which I think you should), it's standards track. If you are talking about a tool provided by you on PyPI, you actually don't need to discuss it with anybody. * Does the functionality described strike the right balance between simplicity and usefulness? It leaves a number of issues still open, so it's hard to tell yet. For example, the 32-bit vs. 64-bit issue should be resolved (which I think comes down to having 4 binaries). Plus, it should talk about installation of multiple copies of the same Python version, e.g. from ActivePython, EPD, etc. * Does the strategy have general support from Martin, who as the person making Python distributions would need to be involved in at least some aspects of it (specifically, having the installer MSI install the launcher and update the file associations to reference it). Notice that the PEP doesn't talk about file associations yet (but it should). I'm fine with the strategy, but I feel that the devil's in the detail. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
I have a trivial little documentation patch for csv.rst. I committed it locally, then I pulled and merged: RDM Note that if you want doc changes to appear in all the current doc RDM sets (2.7, 3.2, dev), then you should start with patching 3.2 and RDM merge it into default, and also patch 2.7, just like any other RDM bug-fix commit. Sorry, I found updating cpython hard enough. I almost gave up on just that one simple change. Anyway, I thought I was supposed to pull from head to 3.2 and 2.7. What is dev? Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot VM exhausted in test_crashers
On Sat, 19 Mar 2011 14:45:10 -0400 David Bolen db3l@gmail.com wrote: The problem is that the failing build is automatically retried when I try to reconnect the buildbot, so it just get rights into trouble again. The comment in that build's changeset looks like it might be getting rolled back? Assuming that's the case, is there a way to flush any pending builds for my slaves before I bring them back online? I've enabled the button to cancel pending builds in the Web UI. You should be able to bring back the Ubuntu slave online. You can do the same (cancel pending builds) for other slaves if needed. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
On Mar 19, 2011, at 11:21 AM, Antoine Pitrou wrote: On Sat, 19 Mar 2011 09:25:07 -0500 s...@pobox.com wrote: The dev guide says something about collapsing changesets. Is that collapsing commits within a changeset or collapsing multiple changesets (whatever that might be)? Do I need this for a trivial change? Can I just push at this point? Once pushed, how does it get merged into the main codebase? Sincerely, I would really recommend that you read a Mercurial tutorial. We could answer all your questions one by one but that wouldn't help you much if you don't understand the concepts. Skip is not alone on this one. I've been using Mercurial for a couple of months now, have read multiple tutorials and whatnot, but am still not clear on how to follow the devguide's suggested cross-branch workflow. The dance of pulling, merging, reverting, collapsing, resolving, null merging, and rebasing cross-linked branches is somewhat more involved and complex than covered in any of the tutorials I've seen. I think if we're going to require a complex workflow, then we're going to have to expect a lot of questions. And those questions shouldn't be brushed-off with go read the tutorial, we have no time for you or words to that effect. RAymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
Antoine == Antoine Pitrou solip...@pitrou.net writes: Antoine On Sat, 19 Mar 2011 09:25:07 -0500 Antoine s...@pobox.com wrote: I have a trivial little documentation patch for csv.rst. I committed it locally, then I pulled and merged: cpython% hg pull pulling from ssh://h...@hg.python.org/cpython searching for changes adding changesets adding manifests adding file changes added 94 changesets with 422 changes to 154 files (+1 heads) Antoine 94 changesets? If you want to avoid risking conflicts, you should hg Antoine pull and hg up (or hg pull -u) before you start working on Antoine something (just like you svn up'ed before working on something). Sorry, this workflow is still new and hugely confusing to me. To make a simple edit to csv.rst I needed to do a couple pulls and merges, local commits, resolve the same conflict multiple times, get rebuffed when I first pushed because there was a tab in the file, and ask a question here. If this is the typical route necessary to make even the simplest changes which would have been a single commit with svn, my already meager rate of contributions is likely to wither away to nothing. The dev guide says something about collapsing changesets. Is that collapsing commits within a changeset or collapsing multiple changesets (whatever that might be)? Do I need this for a trivial change? Can I just push at this point? Once pushed, how does it get merged into the main codebase? Antoine Sincerely, I would really recommend that you read a Mercurial Antoine tutorial. We could answer all your questions one by one but Antoine that wouldn't help you much if you don't understand the Antoine concepts. Thanks for the snide response. If I had time to read an entire book just to learn how to do something I was doing in CVS and Subversion with no trouble, I would have done so by now. I want to understand how to use the workflow set up for this project. If you aren't willing to help me get past my initial difficulties, please don't bother responding. -- Skip Montanaro - s...@pobox.com - http://www.smontanaro.net/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] improvement to declaring abstract properties
I suggested at python-ideas a way that the declaration of abstract properties could be improved to support the decorator syntax: http://mail.python.org/pipermail/python-ideas/2011-March/009411.html . A relatively small change to the property builtin would allow properties to identify themselves as abstract when they are passed abstract methods (the same way that function objects identify themselves as abstract when decorated with @abstractmethod). As a result, @abstractproperty would no longer be needed. I submitted a patch at http://bugs.python.org/issue11610. It includes the changes to the property builtin, documentation, and unit tests. Unfortunately, I have not been able to python-3.3 from a mercurial checkout on either Ubuntu 11.04 or OS X 10.6.6 (for reasons unrelated to the patch), and so I have not been able to test the patch. Darren ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot VM exhausted in test_crashers
Antoine Pitrou solip...@pitrou.net writes: I've enabled the button to cancel pending builds in the Web UI. You should be able to bring back the Ubuntu slave online. You can do the same (cancel pending builds) for other slaves if needed. Thanks. Is that a single button somewhere or do I need to cancel the pending builds individually when there are several waiting? In any event it looks like the retries of the failing 3.x build stopped on their own. I suspect, however, that as soon as something else commits to the 3.x branch and causes a new test, that test_crashers (presumably from changeset 9c6dcb5d8f01) will likely just take the slaves down again. Perhaps the test is just too hard to execute reliably on the entire build farm? -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot VM exhausted in test_crashers
On Sat, 19 Mar 2011 15:41:51 -0400 David Bolen db3l@gmail.com wrote: Thanks. Is that a single button somewhere or do I need to cancel the pending builds individually when there are several waiting? There seems to be a single button. In any event it looks like the retries of the failing 3.x build stopped on their own. Yes, I pressed the button myself :) I suspect, however, that as soon as something else commits to the 3.x branch and causes a new test, that test_crashers (presumably from changeset 9c6dcb5d8f01) will likely just take the slaves down again. The test is already disabled. I don't know if Nick will remove it entirely, but it shouldn't cause any harm in the meantime. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
I think if we're going to require a complex workflow, then we're going to have to expect a lot of questions. And those questions shouldn't be brushed-off with go read the tutorial, we have no time for you or words to that effect. And indeed, I think this (asking questions) is just about the only sensible way to approach this - requests to RTFM are probably less useful than people expect. The only way to learn this stuff is by doing it, not by reading about it. I think I warned that this would happen - that the tools require a large amount of relearning, and that it will take time for committers to adjust. So I'd encourage everybody to keep asking questions, and request that they are answered in a polite manner, even if the one answering thinks that the same question is already answered in some documentation. The only alternative is that some people just give up contributing to Python. I just hope that we can stick with Mercurial for more than five years, before the next hot thing comes along. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hg: inter-branch workflow
On Sat, Mar 19, 2011 at 9:46 PM, Stephen J. Turnbull step...@xemacs.org wrote: To the extent the buildbots are not overloaded, this strategy will indeed save developer time, as most changes are more or less independent of each other (that's why automated merging works at all well), and most of the time something that passes in a branch will also pass after merging. So it's probably easier, and certainly less wearing on other developers, if you detect breakage in the branch rather than waiting for it to happen in the trunk. However, such early detection is not guaranteed because not all semantic conflicts are syntactic conflicts. Ie, the merge may succeed but the code break. Committers would still be obliged to run the tests *locally* before pushing, so it's only cross-platform issues that would potentially slip through the cracks. That may still turn the buildbots red, of course, but the combination should keep them green more reliably. (e.g. in retrospect, I never would have committed test_crashers in its original state if I had easily been able to run it across the buildbot fleet from my sandbox) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows
On Sun, Mar 20, 2011 at 4:49 AM, Martin v. Löwis mar...@v.loewis.de wrote: I, for example, will find issues with it if the implementation uses CreateProcess at some point - the launcher should IMO run Python directly, rather than creating a new process. This, in turn, will probably mean that you cannot actually implement the PEP in Python. runpy performs basically that dance to implement _run_module_as_main(), so it's definitely possible. You just need to be careful with it, since whatever module you invoke is going to inherit the __main__ namespace. The implementation in runpy doesn't suffer that problem, since the interpreter imports it directly, rather than requiring it to bootstrap itself (/tangent: I think I may have just noticed a nasty corner case for PEP 395 to deal with...) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot VM exhausted in test_crashers
On Sun, Mar 20, 2011 at 5:52 AM, Antoine Pitrou solip...@pitrou.net wrote: I suspect, however, that as soon as something else commits to the 3.x branch and causes a new test, that test_crashers (presumably from changeset 9c6dcb5d8f01) will likely just take the slaves down again. The test is already disabled. I don't know if Nick will remove it entirely, but it shouldn't cause any harm in the meantime. Thanks for turning it off (I went to do that myself this morning only to find you had already killed it). I don't want to give up completely on the idea just yet, but I'll experiment in the sandbox before I turn it back on. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows
Am 19.03.2011 23:51, schrieb Nick Coghlan: On Sun, Mar 20, 2011 at 4:49 AM, Martin v. Löwis mar...@v.loewis.de wrote: I, for example, will find issues with it if the implementation uses CreateProcess at some point - the launcher should IMO run Python directly, rather than creating a new process. This, in turn, will probably mean that you cannot actually implement the PEP in Python. runpy performs basically that dance to implement _run_module_as_main(), so it's definitely possible. Does it also support loading a different interpreter version than the one it is running? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
On Sun, Mar 20, 2011 at 5:05 AM, s...@pobox.com wrote: Sorry, this workflow is still new and hugely confusing to me. To make a simple edit to csv.rst I needed to do a couple pulls and merges, local commits, resolve the same conflict multiple times, get rebuffed when I first pushed because there was a tab in the file, and ask a question here. If this is the typical route necessary to make even the simplest changes which would have been a single commit with svn, my already meager rate of contributions is likely to wither away to nothing. Note that some of that isn't unique to the Mercurial transition (specifically, the server side hook that complained about the whitespace change existed in SVN as well). One key difference that we could historically ignore with SVN is that it would happily accept changes from an out-of-date working copy, so long as the committed changes didn't alter any files that had also been updated on the head. In practice, this *did* mean a lot of commits did get rejected, since Misc/NEWS would often conflict. Mercurial's changeset based view of the world means it wants to know how the changesets relate to each other even when they affect different files, unlike SVN which would happily do an implicit merge (creating a state in the central repo that no developer has ever tested locally). So the full flow for a trunk commit in SVN was: svn update # change stuff svn commit -m Whatever # get rejected by server side hooks (e.g. changed files, bad whitespace) make patchcheck # fix whitespace issues svn update # get updated files # Resolve any conflicts, rerun relevant tests svn commit -m Whatever Mercurial isn't really all that different, but it's distributed nature means it want to keep track of even minor things like the local whitespace fixes and the merger of your changes with the other changes that were pushed since you started work. So the example above becomes something like: hg pull -u #change stuff hg commit -m Whatever hg push # get rejected by server side hooks (e.g. changed files, bad whitespace) make patchcheck hg commit -m Fix whitespace hg pull -u hg merge # Resolve any conflicts, rerun relevant tests hg commit -m Merge with remote hg push It definitely produces some additional noise on python-checkins, since things that would normally have been resolved privately by the developer before SVN accepted the commit are now being published to the mailing list. To help with that, I've personally split my python-checkins label into two: one which contains everything from the list, and another which filters out any messages with merge in the subject line. You may also find hg outgoing useful, since that will tell you all the changesets that a call to hg push will publish, rather than just the latest changeset as shown by hg heads. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows
On Sun, Mar 20, 2011 at 9:04 AM, Martin v. Löwis mar...@v.loewis.de wrote: Am 19.03.2011 23:51, schrieb Nick Coghlan: On Sun, Mar 20, 2011 at 4:49 AM, Martin v. Löwis mar...@v.loewis.de wrote: I, for example, will find issues with it if the implementation uses CreateProcess at some point - the launcher should IMO run Python directly, rather than creating a new process. This, in turn, will probably mean that you cannot actually implement the PEP in Python. runpy performs basically that dance to implement _run_module_as_main(), so it's definitely possible. Does it also support loading a different interpreter version than the one it is running? Ouch, no. I guess you're right, any Python based solution will need to create a child process. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
On Sat, Mar 19, 2011 at 12:55 PM, Martin v. Löwis mar...@v.loewis.de wrote: So I'd encourage everybody to keep asking questions, and request that they are answered in a polite manner, even if the one answering thinks that the same question is already answered in some documentation. Thanks. I have some basic questions. The only alternative is that some people just give up contributing to Python. I was about to do just that. :-) Now I've decided to take the plunge instead. I made a test change to the 2.5 README and merged it into 2.6 and 2.7. Then I pushed. It *seems* to have worked, but I also got some traceback: hg push pushing to ssh://h...@hg.python.org/cpython searching for changes remote: adding changesets remote: adding manifests remote: adding file changes remote: added 6 changesets with 6 changes to 1 files remote: change(s) NOT sent, something went wrong: remote: [Failure instance: Traceback from remote host -- Traceback (most recent call last): remote: File /usr/lib/python2.6/dist-packages/twisted/spread/banana.py, line 153, in gotItem remote: self.callExpressionReceived(item) remote: File /usr/lib/python2.6/dist-packages/twisted/spread/banana.py, line 116, in callExpressionReceived remote: self.expressionReceived(obj) remote: File /usr/lib/python2.6/dist-packages/twisted/spread/pb.py, line 514, in expressionReceived remote: method(*sexp[1:]) remote: File /usr/lib/python2.6/dist-packages/twisted/spread/pb.py, line 826, in proto_message remote: self._recvMessage(self.localObjectForID, requestID, objectID, message, answerRequired, netArgs, netKw) remote: --- exception caught here --- remote: File /usr/lib/python2.6/dist-packages/twisted/spread/pb.py, line 840, in _recvMessage remote: netResult = object.remoteMessageReceived(self, message, netArgs, netKw) remote: File /usr/lib/python2.6/dist-packages/twisted/spread/pb.py, line 225, in perspectiveMessageReceived remote: state = method(*args, **kw) remote: File /data/buildbot/master/master.cfg, line 87, in perspective_addChange remote: changedict['category'] = branch_to_category[changedict['branch']] remote: exceptions.KeyError: '2.5' remote: ] remote: notified python-check...@python.org of incoming changeset 3074c77b7121 remote: notified python-check...@python.org of incoming changeset cf5e4bdb24a1 remote: notified python-check...@python.org of incoming changeset 89007356798b remote: notified python-check...@python.org of incoming changeset cc959f114739 remote: notified python-check...@python.org of incoming changeset 69650b81f4b9 remote: notified python-check...@python.org of incoming changeset 5ae4f3bbaebe I also have trouble building the 2.5 branch on OS X Snow Leopard (10.6) with Xcode 3.2 -- this is gcc 4.2.1. I think I'll just give up on that part though, it's probably just too old, and 2.7 builds just fine. I just hope that we can stick with Mercurial for more than five years, before the next hot thing comes along. +1. Just as I hope for the Python 3-4 transition, I hope that whatever comes along next has a better transition strategy. That would make it really hot. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
On Sat, 19 Mar 2011 16:32:53 -0700 Guido van Rossum gu...@python.org wrote: remote: File /data/buildbot/master/master.cfg, line 87, in perspective_addChange remote: changedict['category'] = branch_to_category[changedict['branch']] remote: exceptions.KeyError: '2.5' remote: ] That's because the buildbot notification hook tries to launch new builds on branch 2.5 but it is unknown to the buildmaster. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
On Sun, Mar 20, 2011 at 9:32 AM, Guido van Rossum gu...@python.org wrote: +1. Just as I hope for the Python 3-4 transition, I hope that whatever comes along next has a better transition strategy. That would make it really hot. Given that the hardest part of any transition is updating developers' brains, that would be some pretty damn neat neurohacking right there :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot VM exhausted in test_crashers
Nick Coghlan ncogh...@gmail.com writes: I don't want to give up completely on the idea just yet, but I'll experiment in the sandbox before I turn it back on. If you get to that point again, I'd also be willing to pick a time to manually check out the right branch or whatever and try it manually on one of two of my slaves while I'm watching. Typically I don't pay too much attention to various failures created by testing, but this particular case breaks in a way that the slaves don't recover, so is probably better to put a toe in the water slowly while watching :-) -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
On Sat, Mar 19, 2011 at 12:59 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: On Mar 19, 2011, at 11:21 AM, Antoine Pitrou wrote: On Sat, 19 Mar 2011 09:25:07 -0500 s...@pobox.com wrote: The dev guide says something about collapsing changesets. Is that collapsing commits within a changeset or collapsing multiple changesets (whatever that might be)? Do I need this for a trivial change? Can I just push at this point? Once pushed, how does it get merged into the main codebase? Sincerely, I would really recommend that you read a Mercurial tutorial. We could answer all your questions one by one but that wouldn't help you much if you don't understand the concepts. Skip is not alone on this one. I've been using Mercurial for a couple of months now, have read multiple tutorials and whatnot, but am still not clear on how to follow the devguide's suggested cross-branch workflow. The dance of pulling, merging, reverting, collapsing, resolving, null merging, and rebasing cross-linked branches is somewhat more involved and complex than covered in any of the tutorials I've seen. I think this will always be the case until someone develops a wiki-type code revision system along with a voting model. The most reliable and highest ranking code would be a nucleus whereupon all other revisions from other collaborators would revolve. Links between the revisions and the core would have acceptance requirements. All would be organized around this circular nucleus rather than attempt linearity (which not only requires a benevolent dictator, but that everyone agree so that forks do not create factions -- or worse, abandonment). That being said, it would be a radical adjustment to development style as there would no longer be a canonical or MAIN authority. But whatever, wanna help me built it? Marcos, pangaia.sf.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot VM exhausted in test_crashers
On Sun, Mar 20, 2011 at 9:44 AM, David Bolen db3l@gmail.com wrote: Nick Coghlan ncogh...@gmail.com writes: I don't want to give up completely on the idea just yet, but I'll experiment in the sandbox before I turn it back on. If you get to that point again, I'd also be willing to pick a time to manually check out the right branch or whatever and try it manually on one of two of my slaves while I'm watching. Typically I don't pay too much attention to various failures created by testing, but this particular case breaks in a way that the slaves don't recover, so is probably better to put a toe in the water slowly while watching :-) That would be great. The first thing I'll be trying (once I get back to it) is simply skipping the crasher I think is the main cause of the instability (the compiler recursion one). I already deliberately dodge the two crashers that are known to cause infinite loops, skipping one more wouldn't be a terrible outcome. I need to bug Barry about his PPC buildbot as well. It seemed to be surviving even after I had cranked the expression size in the compiler recursion crasher to absolutely ridiculous levels, so I'm wondering if there may be something different about the way PPC handles the stack. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
Skip wrote: Antoine wrote: 94 changesets? If you want to avoid risking conflicts, you should hg pull and hg up (or hg pull -u) before you start working on something (just like you svn up'ed before working on something). Sorry, this workflow is still new and hugely confusing to me. To make a simple edit to csv.rst I needed to do a couple pulls and merges, local commits, resolve the same conflict multiple times, get rebuffed when I first pushed because there was a tab in the file, and ask a question here. If this is the typical route necessary to make even the simplest changes which would have been a single commit with svn, my already meager rate of contributions is likely to wither away to nothing. This is reminiscent of a message on the Mercurial mailing list recently, to which I responded with a question about people experiencing problems with Mercurial because they are used to file- or directory-specific operations in other systems, eliciting this reply: hg failed saying there were uncommitted changes (his source code changes in another part of the tree). He didn't want to commit those changes yet, they weren't finished. So he was stuck. To his mind, hg was being stupid because it was getting in his way, unnecessarily linking changes in the two sets of files together. The concept of a revision being a state of the whole repo was alien. For most people that concept came in with svn. http://www.selenic.com/pipermail/mercurial/2011-March/037373.html I've spent some time trying to tidy up and improve a document on the Mercurial Wiki about CVS-like working practices with Mercurial, and this might explain the differences in operation between CVS/Subversion and Mercurial, although it doesn't yet distinguish between the narrow file-specific commits you get in systems like CVS and Subversion, and the whole-project commits you get in systems like Mercurial: http://mercurial.selenic.com/wiki/CvsLikePractice (By the time you look at that page, I might have added something, though.) I'm certainly no expert with Mercurial, and I've only been using it as a personal tool since the MoinMoin guys introduced me to it back at EuroPython 2006, so even now the right way or best practice when it comes to propagating fixes between branches/clones is something I'm still figuring out, but having lurked on the recent python-dev threads and having read various queries on the Mercurial list over the past year or so, I get the impression that reaching for things like rebase and mq as the first choice to solve various workflow problems would get some blunt criticism on the Mercurial list. That said, I can't readily find any good guides to managing a multi-branch project, but there seem to be some interesting techniques documented for some of the situations people are likely to encounter. For example: http://mercurial.selenic.com/wiki/DaggyFixes describes pulling fixes selectively which I'll probably have to try on some personal project at some point http://hginit.com/05.html describes the presumably common way of propagating fixes from stable branches to development branches Certainly, the Python devguide is a nice piece of documentation, but I feel that there's an opportunity here to address some documentation issues that would also help others using and adopting Mercurial. For example, submitting clean changes to a project (that collapse thing) is a topic for a document that could usefully be written, containing some nice diagrams that explain the mechanism to newcomers, and it would surely benefit more than just the CPython development effort. Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
I made a test change to the 2.5 README and merged it into 2.6 and 2.7. Then I pushed. It *seems* to have worked, but I also got some traceback: Oops. I wouldn't have expected that tracebacks propagate that far back. This was your hg client talking to the remote hg client executing a post-push hook talking to buildbot raising an exception (for there no being builder category for 2.5). Your changesets were accepted, anyway. I also have trouble building the 2.5 branch on OS X Snow Leopard (10.6) with Xcode 3.2 -- this is gcc 4.2.1. I think I'll just give up on that part though, it's probably just too old, and 2.7 builds just fine. Yes, this is a known limitation. Please don't make any changes to the 2.5 branch unless they are security fixes (changing the copyright is fine, but I'd rather avoid even documentation changes). If you want to build the 2.5 branch, check it out from subversion. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Have we lost changeset info in the buildbots
Le lundi 14 mars 2011 à 15:36 -0400, David Bolen a écrit : Antoine Pitrou solip...@pitrou.net writes: I suggest you try http://code.google.com/p/bbreport/, which provides a very nice command-line interface. Speaking of bbreport, I sometimes use the published page on that site (http://code.google.com/p/bbreport/wiki/PythonBuildbotReport) to check over things, but looking at it today, it seems to most recently have been generated back in January. Or is the generated date line wrong? I ran a cron task to regenerate this page each hour, but I didn't get any feedback and I stopped to use it, so I just removed the cron task. Do you want that I restart this cron task? I don't remember if bbreport can directly generate an HTML page or not. If not, we should patch it :-) Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
Am 20.03.2011 00:39, schrieb Antoine Pitrou: On Sat, 19 Mar 2011 16:32:53 -0700 Guido van Rossum gu...@python.org wrote: remote: File /data/buildbot/master/master.cfg, line 87, in perspective_addChange remote: changedict['category'] = branch_to_category[changedict['branch']] remote: exceptions.KeyError: '2.5' remote: ] That's because the buildbot notification hook tries to launch new builds on branch 2.5 but it is unknown to the buildmaster. Not exactly. It tries to categorize the changes, so that they would should up on a waterfall filtered by category. Since we have no waterfall for 2.5, I didn't add any categorization for it. This should now be fixed. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
In article aanlktiktvxsrmlw0z8ifvz3lkaedv8vui_a4f-oci...@mail.gmail.com, Nick Coghlan ncogh...@gmail.com wrote: Mercurial isn't really all that different, but it's distributed nature means it want to keep track of even minor things like the local whitespace fixes and the merger of your changes with the other changes that were pushed since you started work. So the example above becomes something like: hg pull -u #change stuff hg commit -m Whatever hg push # get rejected by server side hooks (e.g. changed files, bad whitespace) make patchcheck hg commit -m Fix whitespace hg pull -u hg merge # Resolve any conflicts, rerun relevant tests hg commit -m Merge with remote hg push As a side note, if you are prone to accidentally adding extra whitespace (like I am), you can add the whitespace check hook into your local copy of Mercurial so that you will be warned about whitespace problems immediately when you commit things to your local repos rather than later when you try to push them up to the python.org repo. You will need to add checkwhitsepace.py and reindent.py from the official hooks repo into the site-packages directory of the Python instance where your copy of hg is installed and then configure the hook in an hgrc configuration file. On a Unix-y system, here is one way to do it (no warranty on the installation instructions!): # try to avoid permission problems umask 022 hg clone http://hg.python.org/hooks cd hooks # make a dummy setup.py file for distutils cat setup.py EOF1 from distutils.core import setup setup(name='pydevhooks', version='0.1', py_modules=['checkwhitespace', 'reindent'], ) EOF1 # kludge to find the Python instance which your hg uses HGPYTHON=$(head -1 $(which hg) | sed s'/#!//') echo $HGPYTHON # install the hooks sudo $HGPYTHON setup.py install # configure the hooks in your global .hgrc file - affects all hg repos HGRC=~/.hgrc # ... or configure in specific repos #HGRC=/path/to/your_repo/.hg/hgrc cat $HGRC EOF2 [hooks] pretxncommit.whitespace = python:checkwhitespace.check_whitespace_single EOF2 -- Ned Deily, n...@acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows
Thanks for the feedback! On 19/03/2011 7:44 PM, Glenn Linderman wrote: Not all of the ideas below are complementary to each other, some are either or, to allow different thoughts to be inspired or different directions to be taken. Thanks for starting a PEP. On 3/18/2011 11:02 PM, Mark Hammond wrote: The launcher should be as simple as possible (but no simpler.) The launcher could be simpler if it isn't used for launching interactive interpreters as well as script references via Windows associations (more about that after the next quote). The launcher could be simpler if it just read through the file of its first parameter until it finds a line starting with #@ (process as a Windows version of Unix #!) or starting without a # (error case). This avoids the need parse such lines. I don't agree with that at all. The rules for the shebang parsing are very simple - see the man-page I reference in the pep - the file must start with the characters '#!' and the entire command must be 127 chars. The only real parsing needed is to check if the specified command starts with one of 2 fixed strings. I believe this is simpler than parsing multiple lines of the file. Remember, the typical Windows user is not likely to place a #! line in their scripts in the first place, so teaching them what a Unix #! line is, and how the parameter after it should be something that Windows doesn't even use, and the launcher has to work hard to interpret, is not as simple as possible. I disagree with various aspects of that - the typical Windows user is not going to add a shebang or variation of to their Python source files, period. IMO, the kind of user who would is already somewhat likely to know of the shebang convention, so it would not be foreign. Those remaining who are not familiar with it can simply be pointed at existing docs etc for the shebang line and their new knowledge now works on more than Windows (and more than Python on non-Windows platforms) The launcher could be simpler if the Python installer placed versioned Python executables on the PATH. Unfortunately, historically it hasn't. If it did, would, or the launcher installer would place them there for pre-existing versions of Python, then the launcher could work by launching the appropriate version of Python, expecting Windows to find it on the PATH. The PEP doesn't address the level of internal complexity of the launcher necessary to find where Python has been installed, neither for CPython, nor for the alternate implementations to be supported. The PEP intentionally doesn't, but the implementation does - it already does exactly that for CPython. Other implementation may need a different strategy, but we can cross that bridge when we come to it. The launcher could be simpler if a directory \usr\bin were created under Windows Program Files, placed on the PATH, and %ProgramFiles% prepended to the Unix #! line, with the Python/Jython/Cython installers placing appropriately versioned executables in that directory. Could even start a trend for programs ported from Unix. One could even place an env program there, for more simplicity. Again, I disagree - I think in practice the code would be more complex, and having Python assert it owns such directories and executables is a can of worms we should avoid. * When used to launch an interactive Python interpreter, the launcher will support the first command-line argument optionally be a version specifier in the form -n[.n] (where n is a single integer) to nominate a specific version be used. For example, while py.exe may locate and launch the latest Python 2.x implementation installed, a command-line such as py.exe -3 could specify the latest Python 3.x implementation be launched, while py.exe -2.6 could specify Python 2.6 be located and launched. If a Python 2.x implementation is desired to be launched with the -3 flag, the command-line would need to be similar to py.exe -2 -3 (or the specific version of Python could obviously be launched manually without use of this launcher.) I think that a python launcher that is on the PATH that could be used to launch an interactive Python, should be different than one that is used to launch .py[w] scripts. I believe that if you know the Python you want is already on the PATH, you should just use 'python.exe' instead of 'py.exe'. I don't see any reason to use this launcher for interactive Python sessions where this is the case. 1) python should be invoked interactively by typing python or pythonX[.Y] at the CMD prompt, not py. This can be done without a launcher, if appropriate versioned pythons are placed on the PATH. The launcher is really and only needed for handling .py[w] scripts, which, in the Windows way of thinking, can only be associated with one specific, system-wide configured version of Python (presently, the latest one
Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows
On 20/03/2011 4:15 AM, Dj Gilcrease wrote: On Sat, Mar 19, 2011 at 4:44 AM, Glenn Lindermanv+pyt...@g.nevcal.com wrote: 2) If the launcher provides command line options for the benefit of launching interactive Python interpreters, those command line options can have data puns with script names, or can conflict with actual Python options. I believe Python 2 already has a -3 option, for example. FWIW, the point about Python 2 supporting a -3 arg is explicitly mentioned in the PEP. Windows users are not trained that - introduces an option syntax, but rather /. Most _programmer_ users would probably be aware of - as an option syntax, but Python is used as a language for non-programmers in some circles, and few Windows non-programmers understand / much less - and not even command lines very well. So not using a launcher for launching interactive Python sidesteps all that: Python itself is introduced and taught, and no need to teach about (or even have) launcher options that could possibly conflict and confuse, in addition to Python options that may conflict with script names already. (I have seen lots of Windows users use leading punctuation characters on filenames to affect sort order and grouping of files in My Documents, not knowing they can create subdirectories/subfolders, or not wanting to bother with them, since all their applications default to opening things from My Documents.) The command line options I disagree with as well. If the user wants to test a script that has a shebang of python2 with python3 then they should explicitly launch it that way just like you would have to do on *nix See my reply to Glenn on this - the optional -N.N argument must be the first arg, as must the file with the shebang - so if you specify -N.N, no shebang processing is done at all. I expect that in general, the key advantage of shebang parsing will be when the script is launched via file association - eg, entering the command foo.py directly at the command-prompt or double-clicking from explorer. Windows itself will lookup the association and execute py.exe foo.py - there is no opportunity for the user to arrange for arguments to be inserted in that command (only at the tail). If a user explicitly specifies the command py.exe -3 foo.py, I personally think it is fine the shebang line is ignored as the user has expressed a clear intention that a specific version of Python be used. No I think there should be at max 2 environment variables and they should be explicitly set by the user not added by default PYTHON_2 If set would override the latest version of python2 that is launched via a shebang line ending in python or python2 but not python2.x PYTHON_3 if set would override the latest version of python3 that is launched via a shebang line ending in python3 but not python3.x The reference implementation currently supports 3 environment variables - while they have different names, 2 of them work as you describe above. The third is used to nominate how a simple 'python' with no version qualifiers should be treated - I wanted the ability for the simple 'python' to execute a python 3.x version if desired by the user. Thank you Mark for writing up this pep. It is almost identical to the one I was working up but covers more details and actually provides an implementation example. Other then the command line args I agree with everything covered. Thanks for the feedback! Cheers, Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows
On 20/03/2011 5:49 AM, Martin v. Löwis wrote: * Is this really PEP material, or will turning the PEP into a regular spec be suitable? It's PEP material if it is contentious, which I believe it is. Of course it is - this is python-dev wink. I, for example, will find issues with it if the implementation uses CreateProcess at some point - the launcher should IMO run Python directly, rather than creating a new process. This, in turn, will probably mean that you cannot actually implement the PEP in Python. Out of curiosity, what is your objection to having the child process? I guess it must be the system resources needed for the parent process while it waits for the child to terminate so the exit code can be reflected correctly up stream? I would have 3 main problems with dynamically loading Python into the launcher process: * It may be difficult or impossible to do for implementations other than CPython. * There are subtle differences when Python is loaded by an executable in the install directory versus by an executable that is not. Specifically, I'm thinking about the differences in how the default sys.path is populated. * The registry doesn't currently tell us exactly where pythonxx.dll can be located, but does tell is where python.exe is. Thus things would get a little more complex as we go snooping for the location of the DLL. Note that these are almost certainly surmountable, but I wonder if the benefits outweigh the costs. * If it is a PEP, is it Standards Track or Informational? If you are proposing to change the CPython code base (which I think you should), it's standards track. If you are talking about a tool provided by you on PyPI, you actually don't need to discuss it with anybody. Thanks - Standards Track it is then :) FWIW, I have no interest in making this a stand-alone project or distribution. If there is agreement that this shouldn't be part of Python itself (which may well be a reasonable decision to make), I'll let the idea die. * Does the functionality described strike the right balance between simplicity and usefulness? It leaves a number of issues still open, so it's hard to tell yet. For example, the 32-bit vs. 64-bit issue should be resolved (which I think comes down to having 4 binaries). This is an interesting one and one which I don't have a firm opinion about - but the soft opinion I have is something along the lines of the functionality should not be dictated by the bittedness of the launcher itself. IOW, if we decide that a 64bit implementation of Python should be chosen over a 32bit implementation of the exact same version, this should be done by both the 32bit and 64bit launchers. Of course, I welcome more thoughts on this and could be easily swayed. Plus, it should talk about installation of multiple copies of the same Python version, e.g. from ActivePython, EPD, etc. Good point - this is something I haven't considered at all. Any thoughts on this from anyone? Notice that the PEP doesn't talk about file associations yet (but it should). Actually, it does say '...the console version of the launcher should be associated with .py files and the windows version associated with .pyw files.' What extra level of detail do you feel is necessary? I'm fine with the strategy, but I feel that the devil's in the detail. Indeed! But general approval for the strategy is an encouraging first step :) Thanks! Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Avoiding whitespace conflicts (was: I am now lost - committed, pulled, merged, what is collapse?)
Ned Deily n...@acm.org writes: As a side note, if you are prone to accidentally adding extra whitespace (like I am), you can add the whitespace check hook into your local copy of Mercurial so that you will be warned about whitespace problems immediately when you commit things to your local repos rather than later when you try to push them up to the python.org repo. Even earlier that the attempt to commit, one can configure the text editor to show whitespace in order to be aware of it at the point where it's easiest to change: Emacs: M-x whitespace-mode RET Vim: :set list listchars=trail:·,tab:»· Or, instead of all whitespace, highlight trailing whitespace as an error: Emacs: M-: (setq show-trailing-whitespace t) RET Vim: :set list listchars=trail:·,tab:»· :highlight SpecialKey ctermfg=Yellow ctermbg=Red Of course, any Python programmer hoping to comply with PEP 8 is best advised to never insert a U+0009 HORIZONTAL TAB character: Emacs: M-: (setq indent-tabs-mode nil) RET Vim: :set expandtabs One can even tell the editor to remove all the trailing whitespace: Emacs: M-x delete-trailing-whitespace RET Vim: :%s:\s\+$:: More: URL:http://www.emacswiki.org/emacs/WhiteSpace URL:http://stackoverflow.com/questions/3165503/emacs-global-configuration-of-tabs URL:http://vim.wikia.com/wiki/Highlight_unwanted_spaces URL:http://vim.wikia.com/wiki/Remove_unwanted_spaces URL:http://stackoverflow.com/questions/69998/tabs-and-spaces-in-vim This allows a belt-and-braces approach: the repository will reject attempts to commit with whitespace mistakes, but one's text editor can be an ally in never getting to that point. -- \ “For fast acting relief, try slowing down.” —Jane Wagner, via | `\ Lily Tomlin | _o__) | Ben Finney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows
Out of curiosity, what is your objection to having the child process? One of the problems is that parent process will not be able to kill launched script. Simply doing TerminateProcess will kill the launcher, leaving interpreter running. This can be partially fixed with job objects, though. Eugene ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
On Sat, 19 Mar 2011 13:53:11 -0500, s...@pobox.com wrote: I have a trivial little documentation patch for csv.rst. I committed it locally, then I pulled and merged: RDM Note that if you want doc changes to appear in all the current doc RDM sets (2.7, 3.2, dev), then you should start with patching 3.2 and RDM merge it into default, and also patch 2.7, just like any other RDM bug-fix commit. Sorry, I found updating cpython hard enough. I almost gave up on just that one simple change. Anyway, I thought I was supposed to pull from head to 3.2 and 2.7. What is dev? That's doc speak, I guess. http://docs.python.org/dev/library, for example, points to the library reference for the docs on the default branch. -- R. David Murray http://www.bitdance.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hg: inter-branch workflow
On Thu, 17 Mar 2011 14:33:00 +0100, wrote: On Thu, 17 Mar 2011 09:24:26 -0400 R. David Murray rdmurray at bitdance.com wrote: It would be great if rebase did work with share, that would make a push race basically a non-issue for me. rebase as well as strip destroy some history, meaning some of your shared clones may end up having their working copy based on a non-existent changeset. I'm not sure why rebase would be worse that strip in that regard, though. Well, it turns out that this completely doesn't work, though at first it appeared to (and so I pushed). I had a push race, so I did hg pull; hg rebase. Then I looked at the log, and I could (apparently) see my change sets on the top of the stack. So I pushed. Victor then asked why one of my commits deleted Tools/demo/README, and then the next commit restored it. What I was attempting to push was a doc change in 3.1 that I had then merged to 3.2 and default. What I saw when looking closer at the log (after Victor pointed it out) was that my merge commits had lost their parents. I thought that at worst a rebase would screw up my local history, but apparently I managed to push some sort of damaged history. The doc change only got applied to default, since that's the branch I happened to be in when I did the rebase. Needless to say, I'm avoiding rebase henceforth. I don't think this had anything to do with share, by the way, but rather the fact that I had merge commits before I did the rebase, and that rebase apparently operates on branches, not the repository has a whole. It would have been nice if rebase had refused to run given that there were merges, since it clearly doesn't work in that case. I'll backport the doc change tomorrow :( -- R. David Murray http://www.bitdance.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
