[Python-Dev] New update to PEP397 - Python launcher for Windows

[Mark Hammond]

I've updated PEP 397 - "Python launcher for Windows" based on recent 
discussions and Vinay's implementation.


http://www.python.org/dev/peps/pep-0397/ and a copy is attached for your 
convenience.


The main changes are:

* All mentions of the Python reference implementation have been removed 
now that a C implementation exists and is a more accurate implementation 
of the PEP than the Python version.


* Some "implementation details" have been removed and links added to the 
launcher docs.  This was done mainly so the implementation is free to 
make changes based on user feedback and still stay true to the PEP. 
Note that the launcher doc link doesn't exist right now, but should 
magically appear over the next couple of days as Vinay pushes some 
changes I just sent him.


* The recommendation to install into System32 has been changed to a 
recommendation to install directly into \Windows, as the System32 
directory is not on the default path for 32bit processes on a 64bit 
Windows.  Vinay even has a version of an MSI installer which installs 
into this directory.  The PEP also gives installers more leeway on where 
to install the launcher if this directory is not writable.


I think this PEP is getting close to being finalized - please suggest 
whatever changes you feel are necessary ASAP as soon I'll be asking for 
pronouncement.


Thanks - especially to Vinay for taking on the C implementation!

Mark
PEP: 397
Title: Python launcher for Windows
Version: $Revision$
Last-Modified: $Date$
Author: Mark Hammond 
Status: Draft
Type: Standards Track
Content-Type: text/plain
Created: 15-Mar-2011
Post-History: 21-July-2011, 17-May-2011, 15-Mar-2011

Abstract

This PEP describes a Python launcher for the Windows platform.  A 
Python launcher is a single executable which uses a number of 
heuristics to locate a Python executable and launch it with a
specified command line.


Rationale

Windows provides "file associations" so an executable can be associated
with an extension, allowing for scripts to be executed directly in some
contexts (eg., double-clicking the file in Windows Explorer.)  Until now,
a strategy of "last installed Python wins" has been used and while not
ideal, has generally been workable due to the conservative changes in
Python 2.x releases.  As Python 3.x scripts are often syntactically
incompatible with Python 2.x scripts, a different strategy must be used
to allow files with a '.py' extension to use a different executable based
on the Python version the script targets.  This will be done by borrowing
the existing practices of another operating system - scripts will be able
to nominate the version of Python they need by way of a "shebang" line, as
described below.

Unix-like operating systems (referred to simply as "Unix" in this
PEP) allow scripts to be executed as if they were executable images
by examining the script for a "shebang" line which specifies the
actual executable to be used to run the script.  This is described in
detail in the evecve(2) man page [1] and while user documentation will
be created for this feature, for the purposes of this PEP that man
page describes a valid shebang line.

Additionally, these operating systems provide symbolic-links to
Python executables in well-known directories.  For example, many
systems will have a link /usr/bin/python which references a
particular version of Python installed under the operating-system.
These symbolic links allow Python to be executed without regard for
where Python it actually installed on the machine (eg., without
requiring the path where Python is actually installed to be
referenced in the shebang line or in the PATH.)  PEP 394 'The "python"
command on Unix-Like Systems' [2] describes additional conventions
for more fine-grained specification of a particular Python version.

These 2 facilities combined allow for a portable and somewhat 
predictable way of both starting Python interactively and for allowing
Python scripts to execute.  This PEP describes an implementation of a 
launcher which can offer the same benefits for Python on the Windows 
platform and therefore allows the launcher to be the executable
associated with '.py' files to support multiple Python versions
concurrently.

While this PEP offers the ability to use a shebang line which should
work on both Windows and Unix, this is not the primary motivation for
this PEP - the primary motivation is to allow a specific version to be
specified without inventing new syntax or conventions to describe it.

An overview of the launcher.

This PEP outlines the general functionality and key guidelines of a
launcher for Windows.  It is accompanied by an implementation [3],
written in C, which defines the detailed implementation.  Over
time, changes to the implementation may be desired - 

Re: [Python-Dev] Issue10271 - warnings.showwarning should allow any callable object - request commiter

[lekmalek]

On Sun, 17 Jul 2011 19:19:59 -0700
Brett Cannon  wrote:

> Just so people know, I went ahead and fixed this for 3.3 (but not for
> 3.2 since it changes the API in a subtle way).
Yeah, but that shouldn't break anything.

Anyway, thanks a lot for your help, I'm happy it's in 3.3.

lekma

> 
> On Sat, Jul 16, 2011 at 01:33, lekmalek  wrote:
> 
> > Hello all,
> >
> > Can any of you core devs have a look at
> > http://bugs.python.org/issue10271. It seems Brett is really busy
> > right now and this uncontroversial (AFAICT) one liner only needs
> > someone to review it and commit it. The pb is, it's holding me back
> > a little bit, and I really would like to have it in the next 3.2
> > release if possible.
> >
> > Thanks for your help,
> >
> > lekma
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> > http://mail.python.org/mailman/options/python-dev/brett%40python.org
> >

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[Nick Coghlan]

On Thu, Jul 21, 2011 at 12:52 PM, R. David Murray  wrote:
> Indeed.  If I want to run a script with a different python version
> on a unix-like system, I need to know the path to said script.
> We're trying to make python as easy to use on Windows as it is on Unix.
> If find-script-on-path is considered a worthwhile feature, then as Mark
> said it should be added to base Python (on all platforms), not special-cased
> in the Windows launcher.

And given the diverse range of what Python considers to be an
executable script these days, -1000 to that particular feature. Use
`which scriptname`, that's what it's for. The lack of such
functionality in the underpowered cmd shell on Windows isn't Python's
problem to solve - ask MS for a better shell and command line
utilities (assuming Powershell doesn't already offer something
comparable to 'which').

There are reasons I only code specifically for Windows if someone pays
me to do so :P

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[R. David Murray]

On Thu, 21 Jul 2011 10:27:31 +1000, Mark Hammond  
wrote:
> On 21/07/2011 10:18 AM, Glenn Linderman wrote:
> > On 7/20/2011 5:07 PM, Nick Coghlan wrote:
> >> On Thu, Jul 21, 2011 at 8:51 AM, Ethan Furman  wrote:
> >>> I would say that would be a cool enhancement, as it could save a bit of
> >>> typing, but I think the launcher is quite useful even without path
> >>> traversal.
> >> Two related points:
> >>
> >> 1. Walking PATH isn't necessary, but the cwd of the py process should
> >> be inherited from the shell correctly. If it is, then 'py foo.py'
> >> shouldn't need path traversal, it should just look in the current
> >> directory. Using PATHEXT to turn 'foo.py' directly into an executable
> >> command on PATH from any directory is different and out of scope for
> >> the launcher.
> >
> > Sorry, I disagree that it is out of scope.  Looking in the current
> > directory is fine, when the script is there, but my scripts are seldom
> > in my data directories, and I want to run scripts (from the PATH) on
> > data that is in the CWD.  I consider this a _very common_ use case for
> > using scripts/programs, but then if you want to use py from the command
> > line to tweak which version of Python gets used to execute the script
> > (if the default one didn't work, for example, and you want to try a
> > different one), then suddenly, you have to find the path to the script,
> > and specify it explicitly.
> 
> The arguments above apply equally to python.exe.  The launcher's job is 
> to find an appropriate python.exe and launch it, not to locate the 
> scripts and all the command-line parsing that would entail.  If you want 
> this feature you should advocate for it to be added to Python itself and 
> it will then automatically work in the launcher too.

Indeed.  If I want to run a script with a different python version
on a unix-like system, I need to know the path to said script.
We're trying to make python as easy to use on Windows as it is on Unix.
If find-script-on-path is considered a worthwhile feature, then as Mark
said it should be added to base Python (on all platforms), not special-cased
in the Windows launcher.

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: #665194: support roundtripping RFC2822 date stamps in the email.utils module

[Stephen J. Turnbull]

R. David Murray writes:

 > Hexlify, wormaround...our Barry is just full of interesting words :)

Not "full" at all---there's no there there to put them in.

He's a generator!

The FLUFL-always-uses-efficient-idioms-ly y'rs,
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[Eric Snow]

On Wed, Jul 20, 2011 at 7:52 PM, Nick Coghlan  wrote:
> Even better would be for these (and sys.path) to be list subclasses
> that did the right thing under the hood as Glenn suggested. Code that
> *replaces* rather than modifies these attributes would still
> potentially break virtual packages, but code that modifies them in
> place would do the right thing automatically. (Note that all code that
> manipulates sys.path and __path__ attributes requires explicit calls
> to correctly support current namespace package mechanisms, so this
> would actually be an improvement on the status quo rather than making
> anything worse).

+1 as a solution to the problem Glenn brought up.  However, I'm still
not clear on how much code out there changes sys.path in the offending
way, forcing the need to provide a more implicit solution in this PEP
than extend_virtual_paths().  And in cases where sys.path *is*
changed, and it impacts some virtual package, how many places is that
going to happen in one project?  My guess is not many (and so not many
"boilerplate" calls).  Is it worth adding implicit __path__ updates
for that use case, rather than just the extend_virtual_paths()
function?

As an aside, my first reaction to Glenn's suggestion was "that would
be cool".  Would it be a pursuable option?  We can take this over to
import-sig if it is.

-eric
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Indentation in reStructuredText documents (was: [Python-checkins] peps: Restore whitespace characters lost via email transmission.)

[Brian Curtin]

On Wed, Jul 20, 2011 at 20:31, Ben Finney wrote:

> Éric Araujo  writes:
>
> > FYI, reST uses three-space indents, not four (so that blocks align
> > nicely under the leading two dots + one space), so I think the change
> > was intentional.
>
> No, reST doesn't specify any particular level of indentation. Like most
> Python programmers I prefer four-space indentation, so I do the same in
> my reST documents::
>
>..  epigraph::
>“Foo bar baz” —Fnorble
>
> I strongly recommend the same for Python documentation.
>

We already use three and it seems to look and function properly.

-1 to a mass re-spacing.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[Nick Coghlan]

On Thu, Jul 21, 2011 at 9:03 AM, P.J. Eby  wrote:
> Hm.  Yes, there is a way to do something like that, but it would complicate
> things a bit.  We'd need to:
>
> 1. Leave __path__ off of the modules, and always pull them from
> sys.virtual_package_paths, and

Setting __path__ to a sentinel value (imp.VirtualPath?) would break
less code, as hasattr(mod, '__path__') checks would still work.

Even better would be for these (and sys.path) to be list subclasses
that did the right thing under the hood as Glenn suggested. Code that
*replaces* rather than modifies these attributes would still
potentially break virtual packages, but code that modifies them in
place would do the right thing automatically. (Note that all code that
manipulates sys.path and __path__ attributes requires explicit calls
to correctly support current namespace package mechanisms, so this
would actually be an improvement on the status quo rather than making
anything worse).

I'll note that this kind of thing is one of the key reasons the import
state should some day move to a real class - state coherency is one of
the major use cases for the descriptor protocol, which is unavailable
when interdependent state is stored as module attributes. (Don't
worry, that day is a very long way away, if it ever happens at all)

> 2. Before using a value in sys.virtual_package_paths, we'd need to check
> whether sys.path had changed since we last cached anything, and if so, clear
> sys.virtual_package_paths first, to force a refresh.
>
> This doesn't sound particularly forbidding, but there are various unpleasant
> consequences, like being unable to tell whether a module is a package or
> not, and whether it's a virtual package or not.  We'd have to invent new
> ways to denote these things.

Trying to change how packages are identified at the Python level makes
PEP 382 sound positively appealing. __path__ needs to stay :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Indentation in reStructuredText documents (was: [Python-checkins] peps: Restore whitespace characters lost via email transmission.)

[Ben Finney]

Éric Araujo  writes:

> FYI, reST uses three-space indents, not four (so that blocks align
> nicely under the leading two dots + one space), so I think the change
> was intentional.

No, reST doesn't specify any particular level of indentation. Like most
Python programmers I prefer four-space indentation, so I do the same in
my reST documents::

..  epigraph::
“Foo bar baz” —Fnorble

I strongly recommend the same for Python documentation.

-- 
 \ “I was in the first submarine. Instead of a periscope, they had |
  `\   a kaleidoscope. ‘We're surrounded.’” —Steven Wright |
_o__)  |
Ben Finney

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[Mark Hammond]

On 21/07/2011 10:18 AM, Glenn Linderman wrote:

On 7/20/2011 5:07 PM, Nick Coghlan wrote:

On Thu, Jul 21, 2011 at 8:51 AM, Ethan Furman  wrote:

I would say that would be a cool enhancement, as it could save a bit of
typing, but I think the launcher is quite useful even without path
traversal.

Two related points:

1. Walking PATH isn't necessary, but the cwd of the py process should
be inherited from the shell correctly. If it is, then 'py foo.py'
shouldn't need path traversal, it should just look in the current
directory. Using PATHEXT to turn 'foo.py' directly into an executable
command on PATH from any directory is different and out of scope for
the launcher.


Sorry, I disagree that it is out of scope.  Looking in the current
directory is fine, when the script is there, but my scripts are seldom
in my data directories, and I want to run scripts (from the PATH) on
data that is in the CWD.  I consider this a _very common_ use case for
using scripts/programs, but then if you want to use py from the command
line to tweak which version of Python gets used to execute the script
(if the default one didn't work, for example, and you want to try a
different one), then suddenly, you have to find the path to the script,
and specify it explicitly.


The arguments above apply equally to python.exe.  The launcher's job is 
to find an appropriate python.exe and launch it, not to locate the 
scripts and all the command-line parsing that would entail.  If you want 
this feature you should advocate for it to be added to Python itself and 
it will then automatically work in the launcher too.


Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[Mark Hammond]

On 21/07/2011 10:13 AM, Glenn Linderman wrote:

On 7/20/2011 2:43 PM, Glenn Linderman wrote:

It's not py's job to walk the path: the shell does that when you just type
"foo". It locates foo.py, and then invokes py because of file association - py
then checks the file for a shebang to decide which Python to dispatch it to.


Certainly when the launcher is invoked via an association, this would
be the case.  However, when the launcher is invoked via the command
line, then the unqualified name is passed through.  To be useful from
the command line, the launcher should walk the PATH to find the .py file.


The Windows SearchPath API

makes it a pretty easy job to walk the PATH.  Would even allow low cost
additional feature of searching for both   foo and foo.py  at the same time.


But that would also require us to parse the command-line, understand 
which options require 2 args and which just require 1 (making it fragile 
in the face of later versions introducing new options), then 
re-stitching a modified command-line back together for the child 
process.  IMO that is all out of scope.


Are you maybe forgetting about the PATHEXT functionality?  If you 
include .py in your PATHEXT and file associations are set so .py files 
are handled by the launcher, you should be able to directly execute just 
'foo' and have the command processor search the PATH for a foo.py and 
invoke it via the launcher.


Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[Glenn Linderman]

On 7/20/2011 5:11 PM, Mark Hammond wrote:

On 21/07/2011 10:02 AM, Glenn Linderman wrote:

So this tells me that it didn't find a local py.ini (no surprise, I
don't have one) but doesn't tell me that it did find or read
c:\Windows\system32\py.ini much less whether I have the syntax correct
for my [defaults] section.  It doesn't tell me that it didn't find a #!
line (but there isn't one).


I'll have a go at enhancing the debug output for most of the above 
(although note that if it *did* find a shebang line extra output would 
have been generated.)



Perhaps the problem is that there isn't one?  If it finds no virtual
command, maybe it doesn't obey the [defaults] python=3 directive?  The
PEP says it should act like '!#python' (I think the PEP meant
'#!python', though).  There is no " " after '!#python' in the PEP,
so that disqualifies it from being a customized command, there is no
'/usr/bin/' nor '/usr/bin/env ' in front of the 'python' so that means
it is not a virtual command; and it is not a fully-qualified name, so it
doesn't mean that case either looks like the PEP needs a bit of
clarification here.


I'm not sure the PEP needs clarification - possibly just the 
implementation ;)  But let me know if you think otherwise.


Well, at least !# should be changed to #! :)  And then the example of 
"customized command" shows "#! vpython" with the space before the 
vpython, but the text describes the necessary space as being after the 
customized command, at least the way I read it.  So I really don't know 
whether the example is showing an extraneous space after the ! (and not 
one after vpython) or exactly what is meant.





I do have a python on my path, but it is 3.1.2, not 3.2.1 or 2.6.4, and
it runs 2.6.4 as the output shows.


FYI, what is on the path isn't relevant to the launcher.


I didn't expect it to be, I just mentioned it in passing, because the 
launcher doesn't seem to be doing what I expect it to do, but neither 
does it seem to be using the PATH to find a Python either.





And I would expect it to run 3.2.1
with the [defaults] python=3 directive, since that is newer than 3.1.2,
which is on my PATH.


It may be that your copy of the launcher is a little old - some 
changes were pushed just yesterday (but I'm not sure if Vinay made a 
new installer yet).  It has slightly better debug output (although 
generally not what you are asking for yet) and better 
"cross-bittedness" support.


Cheers,

Mark



Mine's a week old, yes.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[Glenn Linderman]

On 7/20/2011 5:07 PM, Nick Coghlan wrote:

On Thu, Jul 21, 2011 at 8:51 AM, Ethan Furman  wrote:

I would say that would be a cool enhancement, as it could save a bit of
typing, but I think the launcher is quite useful even without path
traversal.

Two related points:

1. Walking PATH isn't necessary, but the cwd of the py process should
be inherited from the shell correctly. If it is, then 'py foo.py'
shouldn't need path traversal, it should just look in the current
directory. Using PATHEXT to turn 'foo.py' directly into an executable
command on PATH from any directory is different and out of scope for
the launcher.


Sorry, I disagree that it is out of scope.  Looking in the current 
directory is fine, when the script is there, but my scripts are seldom 
in my data directories, and I want to run scripts (from the PATH) on 
data that is in the CWD.  I consider this a _very common_ use case for 
using scripts/programs, but then if you want to use py from the command 
line to tweak which version of Python gets used to execute the script 
(if the default one didn't work, for example, and you want to try a 
different one), then suddenly, you have to find the path to the script, 
and specify it explicitly.



2. The defined launched command line handling means that "py -m foo"
should also work, so long as the current directory is inherited
correctly.


Even if CWD is a data directory, and foo.py is somewhere on the path?


Cheers,
Nick.



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[Glenn Linderman]

On 7/20/2011 2:43 PM, Glenn Linderman wrote:

It's not py's job to walk the path: the shell does that when you just type
"foo". It locates foo.py, and then invokes py because of file association - py
then checks the file for a shebang to decide which Python to dispatch it to.


Certainly when the launcher is invoked via an association, this would 
be the case.  However, when the launcher is invoked via the command 
line, then the unqualified name is passed through.  To be useful from 
the command line, the launcher should walk the PATH to find the .py file.


The Windows SearchPath API 
 
makes it a pretty easy job to walk the PATH.  Would even allow low cost 
additional feature of searching for both   foo and foo.py  at the same time.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise)

[Mark Hammond]

On 21/07/2011 10:08 AM, Antoine Pitrou wrote:

On Thu, 21 Jul 2011 09:55:28 +1000
Mark Hammond  wrote:


  >  The two proposals

overlap but are not mutually exclusive. For future pythons, 'python33'
is easier to remember and type than 'py -v 3.3' or whatever the proposed
encantation is.


'py -3.3' - less chars to type than 'python33' and with no need to have
every Python directory on your PATH.  IMO it is also simple enough that
people will remember it fairly easily.


Given that Python 2.x has a "-3" option, isn't "py -3.3" kind of
confusing, at least to the eye?


A little, yeah, but IMO practicality beats purity here.  I'd probably 
feel different if I felt 'python -3' was regularly used and would 
continue to be so in the future.  Also, I think most people who would 
potentially use 'python -3' will be aware that running 'py' is a totally 
different command and will adjust accordingly (either by continuing to 
use 'python -3' or adjusting to running 'py -2 -3'.)


The PEP does make explicit mention of this...

Cheers,

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[Mark Hammond]

On 21/07/2011 10:02 AM, Glenn Linderman wrote:

So this tells me that it didn't find a local py.ini (no surprise, I
don't have one) but doesn't tell me that it did find or read
c:\Windows\system32\py.ini much less whether I have the syntax correct
for my [defaults] section.  It doesn't tell me that it didn't find a #!
line (but there isn't one).


I'll have a go at enhancing the debug output for most of the above 
(although note that if it *did* find a shebang line extra output would 
have been generated.)



Perhaps the problem is that there isn't one?  If it finds no virtual
command, maybe it doesn't obey the [defaults] python=3 directive?  The
PEP says it should act like '!#python' (I think the PEP meant
'#!python', though).  There is no " " after '!#python' in the PEP,
so that disqualifies it from being a customized command, there is no
'/usr/bin/' nor '/usr/bin/env ' in front of the 'python' so that means
it is not a virtual command; and it is not a fully-qualified name, so it
doesn't mean that case either looks like the PEP needs a bit of
clarification here.


I'm not sure the PEP needs clarification - possibly just the 
implementation ;)  But let me know if you think otherwise.



I do have a python on my path, but it is 3.1.2, not 3.2.1 or 2.6.4, and
it runs 2.6.4 as the output shows.


FYI, what is on the path isn't relevant to the launcher.


And I would expect it to run 3.2.1
with the [defaults] python=3 directive, since that is newer than 3.1.2,
which is on my PATH.


It may be that your copy of the launcher is a little old - some changes 
were pushed just yesterday (but I'm not sure if Vinay made a new 
installer yet).  It has slightly better debug output (although generally 
not what you are asking for yet) and better "cross-bittedness" support.


Cheers,

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise)

[Antoine Pitrou]

On Thu, 21 Jul 2011 09:55:28 +1000
Mark Hammond  wrote:
> 
>  > The two proposals
> > overlap but are not mutually exclusive. For future pythons, 'python33'
> > is easier to remember and type than 'py -v 3.3' or whatever the proposed
> > encantation is.
> 
> 'py -3.3' - less chars to type than 'python33' and with no need to have 
> every Python directory on your PATH.  IMO it is also simple enough that 
> people will remember it fairly easily.

Given that Python 2.x has a "-3" option, isn't "py -3.3" kind of
confusing, at least to the eye?

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[Nick Coghlan]

On Thu, Jul 21, 2011 at 8:51 AM, Ethan Furman  wrote:
> I would say that would be a cool enhancement, as it could save a bit of
> typing, but I think the launcher is quite useful even without path
> traversal.

Two related points:

1. Walking PATH isn't necessary, but the cwd of the py process should
be inherited from the shell correctly. If it is, then 'py foo.py'
shouldn't need path traversal, it should just look in the current
directory. Using PATHEXT to turn 'foo.py' directly into an executable
command on PATH from any directory is different and out of scope for
the launcher.

2. The defined launched command line handling means that "py -m foo"
should also work, so long as the current directory is inherited
correctly.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[Glenn Linderman]

On 7/20/2011 4:41 PM, Mark Hammond wrote:

On 21/07/2011 7:43 AM, Glenn Linderman wrote:
...


I still get the same behavior.  Is there any debugging output produced
by py.exe that would tell what py.ini it looks in, and what the content
is?  What diagnostic steps might I take to produce additional
information that would help you (or me, along the way) analyze the
issue?  I do not presently have a C compiler installed on this machine,
however, unless it came along for a ride with something else.


It doesn't do exactly what you ask, but setting an environment 
variable PYLAUNCH_DEBUG to any value will cause some debug output to 
be generated to stdout.


Mark



It produces:

d:\>py d:\path\to\foo.py
launcher build: 64bit
launcher executable: Console
File 'C:\Users\Glenn\AppData\Roaming\py.ini' non-existent
maybe_handle_shebang: read 256 bytes
maybe_handle_shebang: BOM not found, using UTF-8
locate_pythons_for_key: unable to open PythonCore key
locate_pythons_for_key: unable to open PythonCore key
run_child: about to run 'C:\Python26\python.exe d:\path\to\foo.py'
  File "d:\path\to\foo.py", line 469
child process exit code: 1

So this tells me that it didn't find a local py.ini (no surprise, I 
don't have one) but doesn't tell me that it did find or read 
c:\Windows\system32\py.ini much less whether I have the syntax correct 
for my [defaults] section.  It doesn't tell me that it didn't find a #! 
line (but there isn't one).


Perhaps the problem is that there isn't one?  If it finds no virtual 
command, maybe it doesn't obey the [defaults] python=3 directive?  The 
PEP says it should act like '!#python' (I think the PEP meant 
'#!python', though).  There is no " " after '!#python' in the PEP, 
so that disqualifies it from being a customized command, there is no 
'/usr/bin/' nor '/usr/bin/env ' in front of the 'python' so that means 
it is not a virtual command; and it is not a fully-qualified name, so it 
doesn't mean that case either looks like the PEP needs a bit of 
clarification here.


I do have a python on my path, but it is 3.1.2, not 3.2.1 or 2.6.4, and 
it runs 2.6.4 as the output shows.  And I would expect it to run 3.2.1 
with the [defaults] python=3 directive, since that is newer than 3.1.2, 
which is on my PATH.


So, still a mystery.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise)

[Mark Hammond]

On 21/07/2011 4:38 AM, Terry Reedy wrote:


Many installers first make an organization directory and then an app
directory within that. This annoys me sometimes when they only have one
app to ever install, but is useful when there might really be multiple
directories, as in our case. (Ditto for start menu entries.) This is
what python should have done a decade ago.


I disagree.  If we followed that advice we would also be in "\Program 
Files".  I have no problem with multiple Python versions installed 
directly off the root, especially given most users probably have a very 
small number of such installations.  I think Python being a developer 
tool rather than a user app is a reasonable justification for that (and 
the justification used when the existing scheme was decided)


> The two proposals

overlap but are not mutually exclusive. For future pythons, 'python33'
is easier to remember and type than 'py -v 3.3' or whatever the proposed
encantation is.


'py -3.3' - less chars to type than 'python33' and with no need to have 
every Python directory on your PATH.  IMO it is also simple enough that 
people will remember it fairly easily.


Also, the launcher supports the ability to select either the 32 or 64bit 
implementation - so maybe 'python33.exe' isn't really good enough and 
should reflect the bittedness?



A python directory also gives a sensible (though optional) place to put
other interpreters and even python-based apps. The launcher does not.


What other interpreters?  IMO it doesn't make sense to have IronPython, 
jython etc be installed there.  Ditto for apps - especially given most 
apps tend to be tied to a subset of all possible Python versions.


Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New tests in stable versions

[Nick Coghlan]

On Thu, Jul 21, 2011 at 8:16 AM, Brett Cannon  wrote:
> I say don't add new tests for the sake of coverage or adding new tests to
> stable branches. Tests for bugfixes are practically required.

I don't *object* to enhanced tests going into maintenance branches,
but the workflow of committing directly to trunk is so much simpler
that I personally would only apply such changes if the new tests
actually uncovered implementation bugs. Then backporting the tests
would useful either as part of fixing the same bugs or else to
demonstrate that the maintenance branch did not have the problem.

So slightly more relaxed than Brett's view:
- definitely apply bug fixes and their tests to affected maintenance branches
- *optionally* apply coverage enhancements to maintenance branches,
but don't feel obliged to do so

I see it as a productivity trade-off - time spent improving coverage
on multiple branches is time not spent on other things. I'm more than
willing to sacrifice some test coverage on the maintenance branches to
get even better test coverage and appropriate new features on trunk
and more bug fixes on maintenance branches.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[Mark Hammond]

On 21/07/2011 7:43 AM, Glenn Linderman wrote:
...


I still get the same behavior.  Is there any debugging output produced
by py.exe that would tell what py.ini it looks in, and what the content
is?  What diagnostic steps might I take to produce additional
information that would help you (or me, along the way) analyze the
issue?  I do not presently have a C compiler installed on this machine,
however, unless it came along for a ride with something else.


It doesn't do exactly what you ask, but setting an environment variable 
PYLAUNCH_DEBUG to any value will cause some debug output to be generated 
to stdout.


Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[Glenn Linderman]

On 7/20/2011 4:03 PM, P.J. Eby wrote:

I'd recommend *always* using it, outside of simple startup code.


So that is a burden on every program.  Documentation would help, but it 
certainly makes updating sys.path much more complex -- 3 lines (counting 
import of pkgutil) instead of one, and the complexity of understanding 
why there is a need for it, when in simple cases the single line works 
fine, but it would be bug prone to have both ways.






So I am still left with my original question:

Is there some way to create a new __path__ that would reflect the 
fact that it has been dynamically created, rather than set from 
__init__.py, and then when it is referenced, calculate (and cache?) 
a new value of __path__ to actually search?


Hm.  Yes, there is a way to do something like that, but it would 
complicate things a bit


From what you said, it would complicate the solution for complex 
packaging tasks, but would return simple extensions of sys.path to being 
simple again.  Sounds like a good tradeoff, but I'll leave that to you 
and other more knowledgeable people to figure out the details and 
implementation... I snipped the explanation, because it is beyond my 
present knowledge base.


Anyway, it seems worth considering.  We just need to sort out what the 
downsides are for any current tools thinking that such modules aren't 
packages.  (But hey, at least it'll be consistent with what such tools 
would think of the on-disk representation!  That is, a tool that 
thinks foo.py alongside a foo/ subdirectory is just a module with no 
package, will also think that 'foo', once imported, is a module with 
no package.)


Please consider it.  I think your initial proposal solves some problems, 
but a version that doesn't complicate the normal, simple, extension of 
sys.path would be a much better solution, so I am happy to hear that you 
have ideas in that regard.  Hopefully, they don't complicate things too 
much more.  So far, I haven't gotten my head around packages as they 
presently exist (this __init__.py stuff seems much more complex than the 
simplicity of Perl imports that I was used to, although I certainly like 
many things about Python better than Perl, and have switched 
whole-heartedly, although I  still have a fair bit of Perl code to port 
in the fullness of time).  I think your proposal here, although 
maintaining some amount of backward-compatibility may require complexity 
of implementation, can simplify the requirements for creating new 
packages, to the extent I understand it.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[P.J. Eby]

At 03:09 PM 7/20/2011 -0700, Glenn Linderman wrote:

On 7/20/2011 6:05 AM, P.J. Eby wrote:

At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote:
When I read about creating __path__ from sys.path, I immediately 
thought of the issue of programs that extend sys.path, and the 
above is the "workaround" for such programs.  but it requires 
such programs to do work, and there are a lot of such programs (I, 
a relative newbie, have had to write some).  As it turns out, I 
can't think of a situation where I have extended sys.path that 
would result in a problem for fancy namespace packages, because so 
far I've only written modules, not packages, and only modules are 
on the paths that I add to sys.path.  But that does not make 
for a general solution.


Most programs extend sys.path in order to import things.  If those 
things aren't yet imported, they don't have a __path__ yet, and so 
don't need to be fixed.  Only programs that modify sys.path 
*after* importing something that has a dynamic __path__ would need 
to do anything about that.


Sure.  But there are a lot of things already imported by Python 
itself, and if this mechanism gets used in the stdlib, a program 
wouldn't know whether it is safe or not, to not bother with the 
pkgutil.extend_virtual_paths() call or not.


I'm not sure I see how the mechanism could meaningfully be used in 
the stdlib, since IIUC we're not going for Perl-style package 
naming.  So, all stdlib packages would be self-contained.



Plus, that requires importing pkgutil, which isn't necessarily done 
by every program that extends the sys.path ("import sys" is 
sufficient at present).


Plus, if some 3rd party packages are imported before sys.path is 
extended, the knowledge of how they are implement is required to 
make a choice about whether it is needed to import pkgutil and call 
extend_virtual_paths or not.


I'd recommend *always* using it, outside of simple startup code.



So I am still left with my original question:

Is there some way to create a new __path__ that would reflect the 
fact that it has been dynamically created, rather than set from 
__init__.py, and then when it is referenced, calculate (and 
cache?) a new value of __path__ to actually search?


Hm.  Yes, there is a way to do something like that, but it would 
complicate things a bit.  We'd need to:


1. Leave __path__ off of the modules, and always pull them from 
sys.virtual_package_paths, and


2. Before using a value in sys.virtual_package_paths, we'd need to 
check whether sys.path had changed since we last cached anything, and 
if so, clear sys.virtual_package_paths first, to force a refresh.


This doesn't sound particularly forbidding, but there are various 
unpleasant consequences, like being unable to tell whether a module 
is a package or not, and whether it's a virtual package or not.  We'd 
have to invent new ways to denote these things.


On the bright side, though, it *would* allow transparent live updates 
to virtual package paths, so it might be worth considering.


By the way, the reason we have to get rid of __path__ is that if we 
kept it, then code could change it, and then we wouldn't know if it 
was actually safe to change it automatically...  even if no code had 
actually changed it.


In principle, we could keep __path__ attributes around, and 
automatically update them in the case where sys.path has changed, so 
long as user code hasn't directly altered or replaced the 
__path__.  But it seems to me to be a dangerous corner case; I'd 
rather that code which touches __path__ be taking responsibility for 
that path's correctness from then on, rather than having it get 
updated (possibly incorrectly) behind its back.


So, I'd say that for this approach, we'd have to actually leave 
__path__ off of virtual packages' parent modules.


Anyway, it seems worth considering.  We just need to sort out what 
the downsides are for any current tools thinking that such modules 
aren't packages.  (But hey, at least it'll be consistent with what 
such tools would think of the on-disk representation!  That is, a 
tool that thinks foo.py alongside a foo/ subdirectory is just a 
module with no package, will also think that 'foo', once imported, is 
a module with no package.)



And, in the absence of knowing (because I didn't write them) whether 
any of the packages I imported before extending sys.path are virtual 
packages or not, I would have to do this every time I extend 
sys.path.  And so it becomes a burden on writing programs.


If the code is so boilerplate as you describe, should sys.path 
become an object that acts like a list, instead of a list, and have 
its append method automatically do the pkgutil.extend_virtual_paths 
for me?  Then I wouldn't have to worry about whether any of the 
packages I imported were virtual packages or not.


Well, then we'd have to worry about other mutation methods, and 
things like 'sys.path = [blah, blah]', as well.  So if we're going to 
dit

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[P.J. Eby]

At 03:22 PM 7/20/2011 -0600, Eric Snow wrote:

On Wed, Jul 20, 2011 at 2:44 PM, P.J. Eby  wrote:
>
> So, yeah, actually, that's looking pretty sweet.  Basically, we 
just have to

> throw a virtual_package_paths dict into the sys module, and do the above
> along with the get_virtual_path() function and add get_subpath() to the
> importer objects, in order to get the PEP's core functionality working.


Exactly.  That's part of why the importlib approach is so appealing to
me.


Actually, it turns out I was a little too optimistic -- the sketch I 
gave doesn't work right for anything but top-level virtual packages, 
because I didn't take into account the part where get_virtual_path() 
needs a parent path.


Fixing *that* error then leads to a really nasty bit of mutual 
recursion in which the parent module imports are attempted over and 
over again in something like O(N**2), I think.  In order to get rid 
of that, _gcd_import would have to grow some internal memoization so 
it doesn't retry the same imports repeatedly.


Ironically enough, this is because _gcd_import() is recursive, and 
thus attempts the imports in the opposite order (sort of) than 
import.c does, which means that you can't get hold of the parent's 
__path__ without recursing (again).  :-(


And trying to work around that with memoization, led me to the 
realization that you actually can't implement PEP 402 using that type 
of recursion.  That is, to implement the spec correctly, _gcd_import 
is going to have to be refactored to iterate left-to-right over 
module name parts, rather than recursing right-to-left.


That's because PEP 402 only allows for processing a virtual path if a 
module is not found, *not* if a module is found but can't be loaded.


But, with importlib currently being recursive, it only knows that a 
parent import failed via ImportError, not whether that error arose 
from failing to find the module, or failing to load the module!


So, the core part of the _gcd_import() function will need to be 
rewritten to iterate instead of recursing.


(Still, it's probably not going to be *terribly* difficult.  I'll 
take a look at doing a sketch of that next, but if I do one I'll send 
it to Import-SIG instead of here; it's not a detail that matters to 
the general PEP discussion.)


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[Ethan Furman]

Glenn Linderman wrote:

  On 7/20/2011 7:19 AM, Vinay Sajip wrote:

It's not py's job to walk the path: the shell does that when you just type
"foo". It locates foo.py, and then invokes py because of file association - py
then checks the file for a shebang to decide which Python to dispatch it to.


Certainly when the launcher is invoked via an association, this would be 
the case.  However, when the launcher is invoked via the command line, 
then the unqualified name is passed through.  To be useful from the 
command line, the launcher should walk the PATH to find the .py file.


I would say that would be a cool enhancement, as it could save a bit of 
typing, but I think the launcher is quite useful even without path 
traversal.


~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New tests in stable versions

[Brett Cannon]

On Wed, Jul 20, 2011 at 11:48, Terry Reedy  wrote:

> On 7/20/2011 12:25 PM, Victor Stinner wrote:
>
>> Le 20/07/2011 17:58, Éric Araujo a écrit :
>>
>>> Do we have a policy of not adding new test files to stable branches?
>>>
>> New logging tests failed during some weeks. If we add new tests, we may
>> also break some stable buildbots. I don't think that we need to add
>> these new tests to a stable version.
>>
>
> When bugs are fixed in stable branches, they are usually accompanied by
> tests that fail without the bugfix. I have understood the policy to be that
> new tests go into stable branches. Failure indicates a bug in either the
> not-really-so-stable branch or the test. In the latter case, remove the test
> everywhere until fixed. In the former case, either fix the bug in the stable
> branch immediately or open an issue and attach the test code (skipping the
> test needed stage) or just disable it and note on the issue that a fix patch
> should re-enable. The logging tests may have been exceptional some way


Right, but Eric is asking about new tests that do nothing more than improve
test coverage, not exercise a fix for a bug.

I say don't add new tests for the sake of coverage or adding new tests to
stable branches. Tests for bugfixes are practically required.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[Glenn Linderman]

On 7/20/2011 6:05 AM, P.J. Eby wrote:

At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote:
When I read about creating __path__ from sys.path, I immediately 
thought of the issue of programs that extend sys.path, and the above 
is the "workaround" for such programs.  but it requires such 
programs to do work, and there are a lot of such programs (I, a 
relative newbie, have had to write some).  As it turns out, I can't 
think of a situation where I have extended sys.path that would result 
in a problem for fancy namespace packages, because so far I've only 
written modules, not packages, and only modules are on the paths that 
I add to sys.path.  But that does not make for a general solution.


Most programs extend sys.path in order to import things.  If those 
things aren't yet imported, they don't have a __path__ yet, and so 
don't need to be fixed.  Only programs that modify sys.path *after* 
importing something that has a dynamic __path__ would need to do 
anything about that.


Sure.  But there are a lot of things already imported by Python itself, 
and if this mechanism gets used in the stdlib, a program wouldn't know 
whether it is safe or not, to not bother with the 
pkgutil.extend_virtual_paths() call or not.


Plus, that requires importing pkgutil, which isn't necessarily done by 
every program that extends the sys.path ("import sys" is sufficient at 
present).


Plus, if some 3rd party packages are imported before sys.path is 
extended, the knowledge of how they are implement is required to make a 
choice about whether it is needed to import pkgutil and call 
extend_virtual_paths  or not.


So I am still left with my original question:



Is there some way to create a new __path__ that would reflect the 
fact that it has been dynamically created, rather than set from 
__init__.py, and then when it is referenced, calculate (and cache?) a 
new value of __path__ to actually search?


That's what extend_virtual_paths() is for.  It updates the __path__ of 
all currently-imported virtual packages.  Where before you wrote:


 sys.path.append('foo')

You would now write:

 sys.path.append('foo')
 pkgutil.extend_virtual_paths('foo')

...assuming you have virtual packages you've already imported.  If you 
don't, there's no reason to call extend_virtual_paths().  But it 
doesn't hurt anything if you call it unnecessarily, because it uses 
sys.virtual_packages to find out what to update, and if you haven't 
imported any virtual packages, there's nothing to update and the call 
will be a quick no-op.


I think I would have to write

sys.path.append('foo')
import pkgutil
pkgutil.extend_virtual_paths('foo')

or I'd get an error.

And, in the absence of knowing (because I didn't write them) whether any 
of the packages I imported before extending sys.path are virtual 
packages or not, I would have to do this every time I extend sys.path.  
And so it becomes a burden on writing programs.


If the code is so boilerplate as you describe, should sys.path become an 
object that acts like a list, instead of a list, and have its append 
method automatically do the pkgutil.extend_virtual_paths for me?  Then I 
wouldn't have to worry about whether any of the packages I imported were 
virtual packages or not.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New tests in stable versions

[Vinay Sajip]

Victor Stinner  haypocalc.com> writes:

> New logging tests failed during some weeks. If we add new tests, we may 
> also break some stable buildbots. I don't think that we need to add 
> these new tests to a stable version.

Just for my information, which logging test failures are you referring to?

Regards,

Vinay Sajip

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: #665194: support roundtripping RFC2822 date stamps in the email.utils module

[R. David Murray]

On Wed, 20 Jul 2011 22:40:17 +0200, Antoine Pitrou  wrote:
> On Wed, 20 Jul 2011 14:12:42 -0400
> Terry Reedy  wrote:
> > On 7/20/2011 11:41 AM, r.david.murray wrote:
> > 
> > > diff --git a/Lib/email/utils.py b/Lib/email/utils.py
> > 
> > >   # We need wormarounds for bugs in these methods in older Pythons (see 
> > > below)
> > 
> > Is 'wormaround' (variation of workaround) an intentional play on the 
> > fact that some worms prey on other 'bugs' ;-? (rather then eating plant 
> > matter?)
> 
> Or perhaps worms dig their way carefully around known bugs?

Hexlify, wormaround...our Barry is just full of interesting words :)

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[Glenn Linderman]

On 7/20/2011 7:19 AM, Vinay Sajip wrote:

Glenn Linderman  g.nevcal.com>  writes:



 Since I don't yet have associations set up that point at the
 launcher, I thought I'd play with saying "py" in front of the
 command.

Why don't you have any associations pointing to the launcher? Did you delete
them? If you uninstall and install the launcher, are they present?


This is a followon from the other day... the launcher installer had 
placed launcher associations in HCU, which are not visible from ftype 
and assoc commands... so I deleted those, which returned my system to 
not having launcher associations, but Python 3.2.1 associations instead 
(they were, and remained, in HLM throughout the install, but HCU 
temporarily overrode them in some circumstances).


So the launcher is installed in c:\windows\system32, but doesn't have 
associations.  Thought I'd play with it from the command line, before 
reinstalling with ALLUSERS=1 (or establishing the associations by hand).


It is still my opinion that the installer should ask whether it should 
be installed for all users or not, but from what you said the other day, 
this may be beyond the .msi installer.






 So I have a command foo.py using Python 3 syntax in a directory on
 my PATH.  It works fine, since Python 3.2.1 is in Python.file. 
 foo.py does not have a #! line.

 I can successfully execute:  foo.py
 However, the following fails:  py foo.py
 It fails, because foo.py is not found.  Instead, I have to specify:
 py d:\path\to\foo.py
 This is annoying, py should walk the PATH for unqualified files (the
 Windows PATH implicitly includes the current directory, of course,

It's not py's job to walk the path: the shell does that when you just type
"foo". It locates foo.py, and then invokes py because of file association - py
then checks the file for a shebang to decide which Python to dispatch it to.


Certainly when the launcher is invoked via an association, this would be 
the case.  However, when the launcher is invoked via the command line, 
then the unqualified name is passed through.  To be useful from the 
command line, the launcher should walk the PATH to find the .py file.





 OK, with that mystery solved, and using Notepad running as
 administrator to actually, successfully edit the file, it still runs
 the wrong version of Python.  Here is the content of the file, what

I'm afraid I can't reproduce this. When I invoke a script with the default
py.ini, py runs it with Python 2. When I add a [defaults]python=3 entry, py
correctly runs it with Python 3.


I still get the same behavior.  Is there any debugging output produced 
by py.exe that would tell what py.ini it looks in, and what the content 
is?  What diagnostic steps might I take to produce additional 
information that would help you (or me, along the way) analyze the 
issue?  I do not presently have a C compiler installed on this machine, 
however, unless it came along for a ride with something else.





 is wrong?  And why is the spacing around the = in the [commands]
 section so inconsistent?

That's just test data, not a real "production" py.ini. I was testing out
something, which is why the spaces around = are every which way, and I never got
around to changing it. More importantly, those customised commands, while
perhaps useful for testing, are useless in everyday Python usage: perhaps -O,
-Werror, -E, -S etc. might be more useful. I'll take suggestions as to what
might be useful customised commands to ship as a default.


Fine. I realize the launcher is still Beta.  This was just a curiosity 
question.



Regards,

Vinay Sajip


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[Eric Snow]

On Wed, Jul 20, 2011 at 2:44 PM, P.J. Eby  wrote:
> At 01:35 PM 7/20/2011 -0600, Eric Snow wrote:
>>
>> This is a really nice solution.  So a virtual package is not imported
>> until a submodule of the virtual package is successfully imported
>
> Correct...
>
>> (except for direct import of pure virtual packages).
>
> Not correct.  ;-)  What we do is avoid creating a parent module or altering
> its __path__ until a submodule/subpackage import is just about to be
> successfully completed.

Good point, though I was talking about direct imports of pure virtual
packages (which you've indicated are disallowed by the current draft).

>> Also, it makes sense that the above applies to all virtual packages,
>> not just pure ones.
>
> Well, if the package isn't "pure" then what you've imported is really just
> an ordinary module, not a package at all.  ;-)

I meant that if the submodule import fails in the "impure" case, the
existing module does not end up with a __path__.

>> When a pure virtual package is directly imported, a new [empty] module
>> is created and its __path__ is set to the matching value in
>> sys.virtual_packages.  However, an "impure" virtual package is not
>> created upon direct import, and its __path__ is not updated until a
>> submodule import is attempted.  Even the sys.virtual_packages entry is
>> not generated until the submodule attempt, since the virtual package
>> mechanism doesn't kick in until the point that an ImportError is
>> currently raised.
>>
>> This isn't that big a deal, but it would be the one behavioral
>> difference between the two kinds of virtual packages.  So either leave
>> that one difference, disallow direct import of pure virtual packages,
>> or attempt to make virtual packages for all non-package imports.  That
>> last one would impose the virtual package overhead on many more
>> imports so it is probably too impractical.  I'm fine with leaving the
>> one difference.
>
> At this point, I've updated the PEP to disallow direct imports of pure
> virtual packages.  AFAICT it's the only approach that ensures you can't get
> false positive imports by having unrelated-but-similarly-named directories
> floating around.

I see what you mean.  That case is probably more important than the
case of having a package that fails to import but submodules of the
package that succeed.

>> FYI, last night I started on an importlib-based implementation for the
>> PEP and the above solution would be really easy to incorporate.
>
> Well, you might want to double-check that now that I've updated the spec.
>  ;-)  In the new approach, you cannot rely on parent modules existing before
> proceeding to the submodule import.
>
> However, I've just glanced at the importlib trunk, and I think I see what
> you mean.  It's already using a recursive approach, rather than an iterative
> one, so the change should be a lot simpler there than in import.c.
>

>
> So, yeah, actually, that's looking pretty sweet.  Basically, we just have to
> throw a virtual_package_paths dict into the sys module, and do the above
> along with the get_virtual_path() function and add get_subpath() to the
> importer objects, in order to get the PEP's core functionality working.


Exactly.  That's part of why the importlib approach is so appealing to
me.  Brett really did a nice job.

-eric
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[P.J. Eby]

At 01:35 PM 7/20/2011 -0600, Eric Snow wrote:

This is a really nice solution.  So a virtual package is not imported
until a submodule of the virtual package is successfully imported


Correct...


(except for direct import of pure virtual packages).


Not correct.  ;-)  What we do is avoid creating a parent module or 
altering its __path__ until a submodule/subpackage import is just 
about to be successfully completed.


See the change I just pushed to the PEP:

   http://hg.python.org/peps/rev/a6f02035c66c

Or read the revised Specification section here (which is a bit easier 
to read than the diff):


   http://www.python.org/dev/peps/pep-0402/#specification

The change is basically that we wait until a successful find_module() 
happens before creating or tweaking any parent modules.  This way, 
the load_module() part will still see an initialized parent package 
in sys.modules, and if it does any relative imports, they'll still work.


(It *does* mean that if an error happens during load_module(), then 
future imports of the virtual package will succeed, but I'm okay with 
that corner case.)




It seems like
sys.virtual_packages should be populated even during a failed
submodule import.  Is that right?


Yes.  In the actual draft, btw, I dubbed it 
``sys.virtual_package_paths`` and made it a dictionary.  This 
actually makes the pkgutil.extend_path() code more general: it'll be 
able to fix the paths of things you haven't actually imported yet.  ;-)




Also, it makes sense that the above applies to all virtual packages,
not just pure ones.


Well, if the package isn't "pure" then what you've imported is really 
just an ordinary module, not a package at all.  ;-)





When a pure virtual package is directly imported, a new [empty] module
is created and its __path__ is set to the matching value in
sys.virtual_packages.  However, an "impure" virtual package is not
created upon direct import, and its __path__ is not updated until a
submodule import is attempted.  Even the sys.virtual_packages entry is
not generated until the submodule attempt, since the virtual package
mechanism doesn't kick in until the point that an ImportError is
currently raised.

This isn't that big a deal, but it would be the one behavioral
difference between the two kinds of virtual packages.  So either leave
that one difference, disallow direct import of pure virtual packages,
or attempt to make virtual packages for all non-package imports.  That
last one would impose the virtual package overhead on many more
imports so it is probably too impractical.  I'm fine with leaving the
one difference.


At this point, I've updated the PEP to disallow direct imports of 
pure virtual packages.  AFAICT it's the only approach that ensures 
you can't get false positive imports by having 
unrelated-but-similarly-named directories floating around.


So, really, there's not a difference, except that you can't import a 
useless empty module that you have no real business importing in the 
first place...  and I'm fine with that.  ;-)




FYI, last night I started on an importlib-based implementation for the
PEP and the above solution would be really easy to incorporate.


Well, you might want to double-check that now that I've updated the 
spec.  ;-)  In the new approach, you cannot rely on parent modules 
existing before proceeding to the submodule import.


However, I've just glanced at the importlib trunk, and I think I see 
what you mean.  It's already using a recursive approach, rather than 
an iterative one, so the change should be a lot simpler there than in import.c.


There probably just needs to be a pair of functions like:

def _get_parent_path(parent):
pmod = sys.modules.get(parent)
if pmod is None:
try:
pmod = _gcd_import(parent)
except ImportError:
# Can't import parent, is it a virtual package?
path = imp.get_virtual_path(parent)
if not path:
# no, allow the parent's import error to propagate
raise
return path
if hasattr(pmod, '__path__'):
return pmod.__path__
else:
return imp.get_virtual_path(parent)

def _get_parent_module(parent):
pmod = sys.modules.get(parent)
if pmod is None:
pmod = sys.modules[parent] = imp.new_module(parent)
if '.' in parent:
head, _, tail = parent.rpartition('.')
setattr(_get_parent_module(head), tail, pmod)
if not hasattr(pmod, '__path__'):
pmod.__path__ = imp.get_virtual_path(parent)
return pmod

And then instead of hanging on to parent_module during the import 
process, you'd just grab a path from _get_parent_path(), and 
initialize parent_module a little later, i.e.:


if parent:
path = _get_parent_path(parent)
if not path:
msg = (_ERR_MSG + '; {} is not

Re: [Python-Dev] [Python-checkins] cpython: #665194: support roundtripping RFC2822 date stamps in the email.utils module

[Antoine Pitrou]

On Wed, 20 Jul 2011 14:12:42 -0400
Terry Reedy  wrote:
> On 7/20/2011 11:41 AM, r.david.murray wrote:
> 
> > diff --git a/Lib/email/utils.py b/Lib/email/utils.py
> 
> >   # We need wormarounds for bugs in these methods in older Pythons (see 
> > below)
> 
> Is 'wormaround' (variation of workaround) an intentional play on the 
> fact that some worms prey on other 'bugs' ;-? (rather then eating plant 
> matter?)

Or perhaps worms dig their way carefully around known bugs?



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[Terry Reedy]

On 7/20/2011 1:04 PM, P.J. Eby wrote:


This part worries me slightly. Imagine a program as such:

datagen.py
json/foo.js
json/bar.js

datagen.py uses the files in json/ to generate sample data for a
database. In datagen.py is the following code:

try:
import json
except ImportError:
import simplejson as json


While reading the PEP, I worried about this standard usage too but 
missed the scenario you imagined. Good catch.



A possible mitigation would be to require that get_subpath() only return
a directory name if that directory in fact contains importable modules
somewhere. This is actually discussed a bit later as an open issue under
"Implementation Notes", indicating that iter_modules() has this issue as
well.


If one actually wants to create a bare-as-possible empty module, one can 
do that now either with a directory containing an empty __init__.py or, 
even cleaner, imp.new_module. So there is no need for the new mechanism 
to ever duplicate either ;-). So +1 on improving back-compatibility.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[Eric Snow]

On Wed, Jul 20, 2011 at 11:04 AM, P.J. Eby  wrote:
> Hm.  Here's another variant that might be easier to implement (even in C),
> and could offer some other advantages as well.
>
> Suppose we replace the sys.virtual_packages set() with a sys.virtual_paths
> dict(): a dictionary that maps from module names to __path__ lists, and
> that's populated by the __path__ creation algorithm described in the PEP.
>  (An empty list would mean that __path__ creation failed for that
> module/package name.)
>
> Now, if a module doesn't have a __path__ (or doesn't exist), we look in
> sys.virtual_paths for the module name.  If the retrieved list is empty, we
> fail the import.  If it's not, we proceed...  but *don't* create a module or
> set the existing module's __path__.
>
> Then, at the point where an import succeeds, and we're going to set an
> attribute on the parent module, we recursively construct parent modules and
> set their __path__ attributes from sys.virtual_paths, if a module doesn't
> exist in sys.path, or its __path__ isn't set.

(I'm guessing you meant sys.modules in that last sentence.)

This is a really nice solution.  So a virtual package is not imported
until a submodule of the virtual package is successfully imported
(except for direct import of pure virtual packages).  It seems like
sys.virtual_packages should be populated even during a failed
submodule import.  Is that right?

Also, it makes sense that the above applies to all virtual packages,
not just pure ones.

>
> Voila.  Now there are fewer introspection problems as well: trying to
> 'import json.foo' when there's no 'foo.py' in any json/ directory will *not*
> create an empty 'json' package in sys.modules as a side-effect.  And it
> won't add a __path__ to the 'json' module if there were a json.py found,
> either.
>
> What's more, since importing a pure virtual package now fails unless you've
> successfully imported something from it before, it makes more sense for it
> to not have a __file__, or a __file__ of None.
>
> Actually, it's too bad that we have to have parent packages in sys.modules,
> or I'd suggest we just make pure virtual packages unimportable, period.

It wouldn't be that hard to disallow their direct import entirely, but
still allow the indirect import when successfully importing a
submodule.  However, that would effectively imply that the import of
submodules of the virtual package will also fail.  In other words, it
may be a source of confusion if a package can't be imported but its
submodule can.

There is one remaining difference between the two types of virtual
packages that's derived from allowing direct import of pure virtual
packages.

When a pure virtual package is directly imported, a new [empty] module
is created and its __path__ is set to the matching value in
sys.virtual_packages.  However, an "impure" virtual package is not
created upon direct import, and its __path__ is not updated until a
submodule import is attempted.  Even the sys.virtual_packages entry is
not generated until the submodule attempt, since the virtual package
mechanism doesn't kick in until the point that an ImportError is
currently raised.

This isn't that big a deal, but it would be the one behavioral
difference between the two kinds of virtual packages.  So either leave
that one difference, disallow direct import of pure virtual packages,
or attempt to make virtual packages for all non-package imports.  That
last one would impose the virtual package overhead on many more
imports so it is probably too impractical.  I'm fine with leaving the
one difference.

>
> Technically, we *could* always create dummy parent modules for virtual
> packages and *not* put them in sys.modules, but I'm not sure if that's a
> good idea.  It would be more consistent in some ways with the idea that
> virtual packages are not directly importable, but an interesting side effect
> would be that if module A does:
>
>  import foo.bar
>
> and module B does:
>
>  import foo.baz
>
> Then module A's version of 'foo' has *only* a 'bar' attribute and B's
> version has *only* a 'baz' attribute.  This could be considered a good
> thing, a bad thing, or a weird thing, depending on how you look at it.  ;-)
>
> Probably, we should stick with the current shared 'foo' instance, even for
> pure virtual packages.  It's just that 'foo' should not exist in
> sys.packages until one of the above imports succeeds.

(Guessing you meant sys.virtual_packages.)

Agreed.

FYI, last night I started on an importlib-based implementation for the
PEP and the above solution would be really easy to incorporate.

-eric

>
> Anyway, thanks for bringing this issue up, because now we can fix the hole
> *entirely*.  If pure virtual packages can never be imported directly, then
> they can *never* create false positive imports -- and the "Backward
> Compatibility" part of the PEP gets shorter.  ;-)
>
> Hurray!  (I'm tempted to run off and tweak the PEP for this right now, but I
> want to see if any of the

Re: [Python-Dev] [Python-checkins] cpython: #665194: support roundtripping RFC2822 date stamps in the email.utils module

[Terry Reedy]

On 7/20/2011 11:41 AM, r.david.murray wrote:


diff --git a/Lib/email/utils.py b/Lib/email/utils.py



  # We need wormarounds for bugs in these methods in older Pythons (see below)


Is 'wormaround' (variation of workaround) an intentional play on the 
fact that some worms prey on other 'bugs' ;-? (rather then eating plant 
matter?)


Terry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] peps: Restore whitespace characters lost via email transmission.

[Georg Brandl]

On 07/20/11 17:05, Éric Araujo wrote:
> Le 20/07/2011 16:40, P.J. Eby a écrit :
>> PEP 12 prescribes four-space indents for PEPs, actually, wherever it
>> mentions an specific indentation depth.
> 
> Ah, thanks.  I see in my docutils docs that David Goodger used three and
> four-space indents (three for indenting the body of a directive, four
> for other cases).
> 
>> (I'm a bit surprised that three-space indents are mandated for
>> anything involving documenting Python in reST, though, since that
>> would mean you'd also have to indent your code samples by three
>> spaces, or else have an editor that supports two different tab widths.)
> 
> I don’t remember this being a pain.  Maybe it’s because the reST I tend
> to edit has much more reST indents than Python code example indents, so
> I don’t mind typing .

Also, chances are that you've tried out your sample code (!) and thus copy
it from a Python file anyway.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New tests in stable versions

[Terry Reedy]

On 7/20/2011 12:25 PM, Victor Stinner wrote:

Le 20/07/2011 17:58, Éric Araujo a écrit :

Do we have a policy of not adding new test files to stable branches?

New logging tests failed during some weeks. If we add new tests, we may
also break some stable buildbots. I don't think that we need to add
these new tests to a stable version.


When bugs are fixed in stable branches, they are usually accompanied by 
tests that fail without the bugfix. I have understood the policy to be 
that new tests go into stable branches. Failure indicates a bug in 
either the not-really-so-stable branch or the test. In the latter case, 
remove the test everywhere until fixed. In the former case, either fix 
the bug in the stable branch immediately or open an issue and attach the 
test code (skipping the test needed stage) or just disable it and note 
on the issue that a fix patch should re-enable. The logging tests may 
have been exceptional some way.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise)

[Terry Reedy]

On 7/20/2011 3:22 AM, Paul Moore wrote:

On 20 July 2011 03:21, Terry Reedy  wrote:

Suppose for Windows there were one '.../python' directory wherever the user
first asks it to be put and that all pythons, not just cpython, are
installed in directories below that and that the small startup file is
copied into or linked from the python directory. Then the one python
directory could be put on the path and left there and never removed by any
python de-installer (unless perhaps it check that there are no subdirs and
*asks* the user.


Hmm. Suppose that directory was "C:\Program Files\Python Launcher" (or
"C:\Windows\system32" if you don't want to add an extra directory to
PATH). And suppose that instead of having a startup file per Python
installation you have a single file called py.exe. Then you have the
launcher!

Plus, the launcher has its own uninstaller, making it a "normal" part
of the Windows environment, rather than being a directory created by
something which doesn't get uninstalled.

Plus, the launcher has a means of dealing with the "generic" python,
python2 and python3 commands, which your proposal doesn't.

Plus, the launcher deals with existing versions of Python, which your
proposal doesn't (except by manual intervention).

But yes, the idea is sound, which is why it's so similar to what Vinay
did with the launcher IMO.


Many installers first make an organization directory and then an app 
directory within that. This annoys me sometimes when they only have one 
app to ever install, but is useful when there might really be multiple 
directories, as in our case. (Ditto for start menu entries.) This is 
what python should have done a decade ago. Now is not too late.


The launcher has to be in a directory somewhere on the path. That 
directory could just as well be 'our' directory. The two proposals 
overlap but are not mutually exclusive. For future pythons, 'python33' 
is easier to remember and type than 'py -v 3.3' or whatever the proposed 
encantation is.


A python directory also gives a sensible (though optional) place to put 
other interpreters and even python-based apps. The launcher does not.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream)

[Eric Snow]

On Wed, Jul 20, 2011 at 12:03 PM, Kerrick Staley  wrote:
> On Wed, Jul 20, 2011 at 3:28 AM, Nick Coghlan  wrote:
>> Done: http://www.python.org/dev/peps/pep-0394/
>
> Quick question: When I do "svn up", it doesn't show any changes. Has
> it been switched over to Mercurial recently?

http://hg.python.org/peps/

-eric

>
> Thanks,
> Kerrick Staley
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream)

[Kerrick Staley]

On Wed, Jul 20, 2011 at 3:28 AM, Nick Coghlan  wrote:
> Done: http://www.python.org/dev/peps/pep-0394/

Quick question: When I do "svn up", it doesn't show any changes. Has
it been switched over to Mercurial recently?

Thanks,
Kerrick Staley
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[P.J. Eby]

At 12:37 PM 7/20/2011 -0400, Erik wrote:

The best solution I can think of would be to have a way for a module
to mark itself as "finalized" (I'm not sure if that's the best
term--just the first that popped into my head).  This would prevent
its __path__ from being created or extended in any way.  For example,
if the json module contains `__finalized__ = True` or something of the
like, any `import json.foo` would immediately fail.


That wouldn't actually fix the problem Jeff brought up, which was the 
case where there *wasn't* a json.py.


In any case, we can fix this now by banning direct import of 
pure-virtual packages.




In that case there would need to be a way to mark a directory as not
containing importable code.  Not sure what the best approach to that
would be, especially since one of the goals of this PEP seems to be to
avoid marker files.


For this particular issue, we don't need it.  For tools that process 
Python code, or use pkgutil.walk_modules(), there may still be use 
cases, so we'll keep an eye open for relevant input.  Hopefully 
someone will say something that jars loose an idea or two, as 
happened with Jeff's issue above.


(Btw, as we speak, I am swiping Jeff's example and adding it into the 
PEP.  ;-)  It makes a great motivating example for banning 
pure-virtual package imports.)


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[P.J. Eby]

At 08:56 AM 7/20/2011 -0700, Jeff Hardy wrote:

On Tue, Jul 19, 2011 at 8:58 PM, P.J. Eby  wrote:
> The biggest likely exception to the above would be when a piece of
> code tries to check whether some package is installed by importing
> it.  If this is done *only* by importing a top-level module (i.e., not
> checking for a ``__version__`` or some other attribute), *and* there
> is a directory of the same name as the sought-for package on
> ``sys.path`` somewhere, *and* the package is not actually installed,
> then such code could *perhaps* be fooled into thinking a package is
> installed that really isn't.

This part worries me slightly. Imagine a program as such:

datagen.py
json/foo.js
json/bar.js

datagen.py uses the files in json/ to generate sample data for a
database. In datagen.py is the following code:

try:
import json
except ImportError:
import simplejson as json

Currently, this works just fine, but if will break (as I understand
it) under the PEP because the json directory will become a virtual
package and no ImportError will be raised.


Well, it won't fail as long if there actually *is* a json module or 
package on the path.  ;-)  But I do see your point.




Is there a mitigation for this in the PEP that I've missed?


A possible mitigation would be to require that get_subpath() only 
return a directory name if that directory in fact contains importable 
modules somewhere.  This is actually discussed a bit later as an open 
issue under "Implementation Notes", indicating that iter_modules() 
has this issue as well.


The main open questions in doing this kind of checking have to do 
with recursion: it's perfectly valid to have say, a 'zc/' directory 
whose only content is a 'buildout/' subdirectory.


Of course, it still wouldn't help if the 'json/' subdirectory in your 
example did contain .py files.


There is another possibility, though:

What if we change the logic for pure-virtual package creation so that 
the parent module is created *if and only if* a child module is found?


In that case, trying to import a pure virtual 'zc' package would 
fail, but importing 'zc.buildout' would succeed as long as there was 
a zc/buildout.py or a zc/buildout/__init__.py somewhere.


And in your example, 'import json' would fail -- which is to say, succeed.  ;-)

This is a minor change to the spec, though perhaps a bit hairier to 
implement in practice.


The current import.c loop over the module name parts (iterating over 
say, 'zc', then 'buildout', and importing them in turn) would need to 
be reworked so that it could either roll back the virtual package 
creation in the event of sub-import failure or conversely delay 
creation of the parent package(s) until a sub-import finds a module.


I certainly think it's *doable*, mind you, but I'd hate to have to do 
it in C.  ;-)


Hm.  Here's another variant that might be easier to implement (even 
in C), and could offer some other advantages as well.


Suppose we replace the sys.virtual_packages set() with a 
sys.virtual_paths dict(): a dictionary that maps from module names to 
__path__ lists, and that's populated by the __path__ creation 
algorithm described in the PEP.  (An empty list would mean that 
__path__ creation failed for that module/package name.)


Now, if a module doesn't have a __path__ (or doesn't exist), we look 
in sys.virtual_paths for the module name.  If the retrieved list is 
empty, we fail the import.  If it's not, we proceed...  but *don't* 
create a module or set the existing module's __path__.


Then, at the point where an import succeeds, and we're going to set 
an attribute on the parent module, we recursively construct parent 
modules and set their __path__ attributes from sys.virtual_paths, if 
a module doesn't exist in sys.path, or its __path__ isn't set.


Voila.  Now there are fewer introspection problems as well: trying to 
'import json.foo' when there's no 'foo.py' in any json/ directory 
will *not* create an empty 'json' package in sys.modules as a 
side-effect.  And it won't add a __path__ to the 'json' module if 
there were a json.py found, either.


What's more, since importing a pure virtual package now fails unless 
you've successfully imported something from it before, it makes more 
sense for it to not have a __file__, or a __file__ of None.


Actually, it's too bad that we have to have parent packages in 
sys.modules, or I'd suggest we just make pure virtual packages 
unimportable, period.


Technically, we *could* always create dummy parent modules for 
virtual packages and *not* put them in sys.modules, but I'm not sure 
if that's a good idea.  It would be more consistent in some ways with 
the idea that virtual packages are not directly importable, but an 
interesting side effect would be that if module A does:


  import foo.bar

and module B does:

  import foo.baz

Then module A's version of 'foo' has *only* a 'bar' attribute and B's 
version has *only* a 'baz' attribute.  This could be consider

Re: [Python-Dev] New tests in stable versions

[Victor Stinner]

Le 20/07/2011 17:58, Éric Araujo a écrit :

Do we have a policy of not adding new test files to stable branches?
New logging tests failed during some weeks. If we add new tests, we may 
also break some stable buildbots. I don't think that we need to add 
these new tests to a stable version.


Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[Erik]

On Wed, Jul 20, 2011 at 11:56 AM, Jeff Hardy  wrote:
> On Tue, Jul 19, 2011 at 8:58 PM, P.J. Eby  wrote:
>> The biggest likely exception to the above would be when a piece of
>> code tries to check whether some package is installed by importing
>> it.  If this is done *only* by importing a top-level module (i.e., not
>> checking for a ``__version__`` or some other attribute), *and* there
>> is a directory of the same name as the sought-for package on
>> ``sys.path`` somewhere, *and* the package is not actually installed,
>> then such code could *perhaps* be fooled into thinking a package is
>> installed that really isn't.
>
> This part worries me slightly. Imagine a program as such:
>
> datagen.py
> json/foo.js
> json/bar.js
>
> datagen.py uses the files in json/ to generate sample data for a
> database. In datagen.py is the following code:
>
> try:
>    import json
> except ImportError:
>    import simplejson as json
>
> Currently, this works just fine, but if will break (as I understand
> it) under the PEP because the json directory will become a virtual
> package and no ImportError will be raised. Is there a mitigation for
> this in the PEP that I've missed?

This problem was brought up a few times on import-sig, but I don't
think a solution was ever decided on.

The best solution I can think of would be to have a way for a module
to mark itself as "finalized" (I'm not sure if that's the best
term--just the first that popped into my head).  This would prevent
its __path__ from being created or extended in any way.  For example,
if the json module contains `__finalized__ = True` or something of the
like, any `import json.foo` would immediately fail.

Of course, this would put all the onus on the json module to solve
this problem, and other modules might actually wish to be extendable
into packages, in which case you'd still have this problem.  In that
case there would need to be a way to mark a directory as not
containing importable code.  Not sure what the best approach to that
would be, especially since one of the goals of this PEP seems to be to
avoid marker files.

Erik
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] New tests in stable versions

[Éric Araujo]

Hello everyone,

I’ve seen recent commits in the default branch (3.3) that improve test
coverage (for example logging) or add new test files (cgitb, committed
by Brian).  Do we have a policy of not adding new test files to stable
branches?  For existing test files that get more tests, is the commit to
stable branches left to the decision of the committer, or should stable
versions get the new tests too?

Regards
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[Jeff Hardy]

On Tue, Jul 19, 2011 at 8:58 PM, P.J. Eby  wrote:
> The biggest likely exception to the above would be when a piece of
> code tries to check whether some package is installed by importing
> it.  If this is done *only* by importing a top-level module (i.e., not
> checking for a ``__version__`` or some other attribute), *and* there
> is a directory of the same name as the sought-for package on
> ``sys.path`` somewhere, *and* the package is not actually installed,
> then such code could *perhaps* be fooled into thinking a package is
> installed that really isn't.

This part worries me slightly. Imagine a program as such:

datagen.py
json/foo.js
json/bar.js

datagen.py uses the files in json/ to generate sample data for a
database. In datagen.py is the following code:

try:
import json
except ImportError:
import simplejson as json

Currently, this works just fine, but if will break (as I understand
it) under the PEP because the json directory will become a virtual
package and no ImportError will be raised. Is there a mitigation for
this in the PEP that I've missed?

> However, even in the rare case where all these conditions line up to
> happen at once, the failure is more likely to be annoying than
> damaging.  In most cases, after all, the code will simply fail a
> little later on, when it actually tries to DO something with the
> imported (but empty) module.  (And code that checks ``__version__``
> attributes or for the presence of some desired function, class, or
> module in the package will not see a false positive result in the
> first place.)

It may only be annoying, but it's still a breaking change, and a
subtle one at that. Checking __version__ is of course possible, but
it's never been necessary before, so it's unlikely there's much code
that does it. It also makes the fallback code significantly less neat.

- Jeff
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] peps: Restore whitespace characters lost via email transmission.

[Éric Araujo]

Le 20/07/2011 16:40, P.J. Eby a écrit :
> PEP 12 prescribes four-space indents for PEPs, actually, wherever it
> mentions an specific indentation depth.

Ah, thanks.  I see in my docutils docs that David Goodger used three and
four-space indents (three for indenting the body of a directive, four
for other cases).

> (I'm a bit surprised that three-space indents are mandated for
> anything involving documenting Python in reST, though, since that
> would mean you'd also have to indent your code samples by three
> spaces, or else have an editor that supports two different tab widths.)

I don’t remember this being a pain.  Maybe it’s because the reST I tend
to edit has much more reST indents than Python code example indents, so
I don’t mind typing .

Cheers
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] peps: Restore whitespace characters lost via email transmission.

[P.J. Eby]

At 04:21 PM 7/20/2011 +0200, Éric Araujo wrote:

FYI, reST uses three-space indents, not four (so that blocks align
nicely under the leading two dots + one space), so I think the change
was intentional.  The “Documenting Python” guide tells this (in the
standard docs), and I think it applies to PEPs too.


PEP 12 prescribes four-space indents for PEPs, actually, wherever it 
mentions an specific indentation depth.  Also, a formfeed character 
was lost, not just the leading spaces.


Essentially, though, I was just merging my working copy, and those 
were the only differences that showed up (apart from the filled-in 
Post-History header), so I assumed it was just whitespace lost in transmission.


(I'm a bit surprised that three-space indents are mandated for 
anything involving documenting Python in reST, though, since that 
would mean you'd also have to indent your code samples by three 
spaces, or else have an editor that supports two different tab widths.)  


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[P.J. Eby]

At 10:40 AM 7/20/2011 -0400, Neal Becker wrote:
I wonder if this fixes the long-standing issue in OS vendor's 
distributions.  In

Fedora, for example, there is both arch-specific and non-arch directories:
/usr/lib/python2.7 + /usr/lib64/python2.7, for example.  Pure python 
goes into
/usr/lib/python2.7, and code including binaries goes into 
/usr/lib64/python2.7.
But if a package has both, it all has to go into 
/usr/lib64/python2.7, because

the current loader can't find pieces in 2 different directories.

You can't have both /usr/lib/python2.7/site-packages/foo and
/usr/lib64/python2.7/site-packages/foo.

So if this PEP will allow pieces of foo to be found in 2 different 
places, that

would be helpful, IMO.


It's more of a long-term solution than a short-term one.  In order 
for it to work the way you want, 'foo' would need to have its main 
code in foo.py rather than foo/__init__.py.


You could of course make that change on the author's behalf for your 
distro, or remove it altogether if it doesn't contain any actual 
code.  However, if you're going to make changes, you could change its 
__init__.py right now to append extra directories to the module 
__path__...  and that's something you can do right now.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[Neal Becker]

I wonder if this fixes the long-standing issue in OS vendor's distributions.  
In 
Fedora, for example, there is both arch-specific and non-arch directories: 
/usr/lib/python2.7 + /usr/lib64/python2.7, for example.  Pure python goes into 
/usr/lib/python2.7, and code including binaries goes into /usr/lib64/python2.7. 
 
But if a package has both, it all has to go into /usr/lib64/python2.7, because 
the current loader can't find pieces in 2 different directories.

You can't have both /usr/lib/python2.7/site-packages/foo and 
/usr/lib64/python2.7/site-packages/foo.

So if this PEP will allow pieces of foo to be found in 2 different places, that 
would be helpful, IMO.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] peps: Restore whitespace characters lost via email transmission.

[Éric Araujo]

Hi,

> changeset:   3903:728660b53208
> user:pje
> date:Wed Jul 20 09:56:48 2011 -0400
> summary:
>   Restore whitespace characters lost via email transmission.
> [...]
> diff --git a/pep-0402.txt b/pep-0402.txt
> --- a/pep-0402.txt
> +++ b/pep-0402.txt
> @@ -38,13 +38,13 @@
>  
>  .. epigraph::
>  
> -   "Most packages are like modules.  Their contents are highly
> [snip]
> +"Most packages are like modules.  Their contents are highly

FYI, reST uses three-space indents, not four (so that blocks align
nicely under the leading two dots + one space), so I think the change
was intentional.  The “Documenting Python” guide tells this (in the
standard docs), and I think it applies to PEPs too.

Regards
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[Vinay Sajip]

Glenn Linderman  g.nevcal.com> writes:


> Since I don't yet have associations set up that point at the
> launcher, I thought I'd play with saying "py" in front of the
> command.

Why don't you have any associations pointing to the launcher? Did you delete
them? If you uninstall and install the launcher, are they present?

> So I have a command foo.py using Python 3 syntax in a directory on
> my PATH.  It works fine, since Python 3.2.1 is in Python.file. 
> foo.py does not have a #! line.
> I can successfully execute:  foo.py
> However, the following fails:  py foo.py
> It fails, because foo.py is not found.  Instead, I have to specify:
> py d:\path\to\foo.py
> This is annoying, py should walk the PATH for unqualified files (the
> Windows PATH implicitly includes the current directory, of course,

It's not py's job to walk the path: the shell does that when you just type
"foo". It locates foo.py, and then invokes py because of file association - py
then checks the file for a shebang to decide which Python to dispatch it to.

> OK, with that mystery solved, and using Notepad running as
> administrator to actually, successfully edit the file, it still runs
> the wrong version of Python.  Here is the content of the file, what

I'm afraid I can't reproduce this. When I invoke a script with the default
py.ini, py runs it with Python 2. When I add a [defaults]python=3 entry, py 
correctly runs it with Python 3.

> is wrong?  And why is the spacing around the = in the [commands]
> section so inconsistent?

That's just test data, not a real "production" py.ini. I was testing out
something, which is why the spaces around = are every which way, and I never got
around to changing it. More importantly, those customised commands, while
perhaps useful for testing, are useless in everyday Python usage: perhaps -O,
-Werror, -E, -S etc. might be more useful. I'll take suggestions as to what
might be useful customised commands to ship as a default.

Regards,

Vinay Sajip

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[R. David Murray]

On Tue, 19 Jul 2011 23:58:55 -0400, "P.J. Eby"  wrote:
> Worse, this is not just a problem for new users: it prevents *anyone*
> from easily splitting a package into separately-installable
> components.  In Perl terms, it would be as if every possible ``Net::``
> module on CPAN had to be bundled up and shipped in a single tarball!

In general the simplicity of the proposed mechanism and implementation
is attractive.  However, this bit of discussion struck me as sending the
wrong message.  We don't *want* something like the CPAN module
hierarchy.  I prefer to keep things as flat as practical.  Namespace
packages clearly have utility, but please let's not descend into
java-esq package hierarchies.

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[Eric V. Smith]

On 07/20/2011 08:57 AM, P.J. Eby wrote:
> At 06:46 PM 7/20/2011 +1000, Nick Coghlan wrote:
>> On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby  wrote:
>> > So, without further ado, here it is:
>>
>> I pushed this version up to the PEPs repo, so it now has a number
>> (402) and can be read in prettier HTML format:
>> http://www.python.org/dev/peps/pep-0402/
> 
> Technically, shouldn't this be a 3XXX series PEP?  Or are we not doing
> those any more now that all PEPs would be 3XXX?

I think we're back to normal PEP numbering. PEP 382 was also 3.x only.

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[P.J. Eby]

At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote:
When I read about creating __path__ from sys.path, I immediately 
thought of the issue of programs that extend sys.path, and the above 
is the "workaround" for such programs.  but it requires such 
programs to do work, and there are a lot of such programs (I, a 
relative newbie, have had to write some).  As it turns out, I can't 
think of a situation where I have extended sys.path that would 
result in a problem for fancy namespace packages, because so far 
I've only written modules, not packages, and only modules are on the 
paths that I add to sys.path.  But that does not make for a general solution.


Most programs extend sys.path in order to import things.  If those 
things aren't yet imported, they don't have a __path__ yet, and so 
don't need to be fixed.  Only programs that modify sys.path *after* 
importing something that has a dynamic __path__ would need to do 
anything about that.



Is there some way to create a new __path__ that would reflect the 
fact that it has been dynamically created, rather than set from 
__init__.py, and then when it is referenced, calculate (and cache?) 
a new value of __path__ to actually search?


That's what extend_virtual_paths() is for.  It updates the __path__ 
of all currently-imported virtual packages.  Where before you wrote:


 sys.path.append('foo')

You would now write:

 sys.path.append('foo')
 pkgutil.extend_virtual_paths('foo')

...assuming you have virtual packages you've already imported.  If 
you don't, there's no reason to call extend_virtual_paths().  But it 
doesn't hurt anything if you call it unnecessarily, because it uses 
sys.virtual_packages to find out what to update, and if you haven't 
imported any virtual packages, there's nothing to update and the call 
will be a quick no-op.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[P.J. Eby]

At 06:46 PM 7/20/2011 +1000, Nick Coghlan wrote:

On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby  wrote:
> So, without further ado, here it is:

I pushed this version up to the PEPs repo, so it now has a number
(402) and can be read in prettier HTML format:
http://www.python.org/dev/peps/pep-0402/


Technically, shouldn't this be a 3XXX series PEP?  Or are we not 
doing those any more now that all PEPs would be 3XXX?


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[Paul Moore]

On 20 July 2011 10:17, Glenn Linderman  wrote:
> However, the following fails:  py foo.py
> It fails, because foo.py is not found.  Instead, I have to specify: py
> d:\path\to\foo.py
> This is annoying, py should walk the PATH for unqualified files (the Windows
> PATH implicitly includes the current directory, of course, but if it were in
> the current directory, it would just work).

Just as a reference point:

PS 12:26 D:\Data
>type .\tt.py
import sys
print sys.argv
PS 12:26 D:\Data
>py tt.py
['tt.py']

This is Windows XP 32-bit, using Powershell as my shell. Also works in cmd.exe.

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.2.1 encoding surprise

[Glenn Linderman]

On 7/18/2011 6:41 PM, Vinay Sajip wrote:

>  So I can fix my machine, now that I understand what went wrong
>   (delete py.exe entries from HCU, and put them in HLM instead).
>   Then the other problem I have, is why py.exe launched py 2.6.4
>   instead of py 3.2.1 when 3.2.1 is newer, and I don't have a #!
>   line.  That is probably the defined behavior of the launcher, to
>   prefer 2.x if 3 isn't specified.  I'll have to reread the launcher
>   PEP.  In my environment, I would much prefer to use 3.x if 2.x isn't
>   specified... so hopefully when I reread the launcher PEP, I'll
>   discover a way to do that.  (I recall one can put -3 on the command
>   line, so I could do that in the py.exe ftype commands, but hopefully
>   there is a way to tweak the py.ini to do that.)

Yes, the PEP explains what you need to do to get Python 3 to be the default:
use the py.ini file or use an environment variable.

The use of py from the command line is merely a convenience for developers (as
the PEP says) - it's better to rely on shebang lines together with settings in
the .ini to get the behaviour you want.


OK, took a few minutes to play with the launcher again.  I have removed 
the HCU classes that point at the launcher, leaving me with all my 
Python.File stuff pointing at some version of Python or another.  I seem 
to have Python 2.6.4, Python 3.1, and Python 3.2.1 installed, all 64-bit.


py.exe is located in c:\Windows\system32.exe, I modified the empty 
py.ini that was installed to contain


Since I don't yet have associations set up that point at the launcher, I 
thought I'd play with saying "py" in front of the command.


So I have a command foo.py using Python 3 syntax in a directory on my 
PATH.  It works fine, since Python 3.2.1 is in Python.file.  foo.py does 
not have a #! line.


I can successfully execute:  foo.py

However, the following fails:  py foo.py
It fails, because foo.py is not found.  Instead, I have to specify: py 
d:\path\to\foo.py
This is annoying, py should walk the PATH for unqualified files (the 
Windows PATH implicitly includes the current directory, of course, but 
if it were in the current directory, it would just work).


So when I: type c:\windows\system32\py.ini
I get a set of h2 h3 v2 v3 commands in a section called [commands].  Not 
sure where my [defaults] went.  Is there some more Windows magic to be 
explained?  WOW6432-ness, perhaps?  Probably my editor is running as a 
32-bit process, so looked in the wrong C:\windows\system32 !!


OK, with that mystery solved, and using Notepad running as administrator 
to actually, successfully edit the file, it still runs the wrong version 
of Python.  Here is the content of the file, what is wrong?  And why is 
the spacing around the = in the [commands] section so inconsistent?


[defaults]
python=3

[commands]
h2=c:\Python26\python -h
h3 = c:\Python32\python -h
v2= c:\Python26\python -V
v3 =c:\Python32\python -V


OK, so when I specify: py d:\path\to\foo.py
It fails, but this time it is because it launches Python 2.6.4.  So the 
[defaults] section doesn't seem to have an effect.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[Glenn Linderman]

On 7/19/2011 8:58 PM, P.J. Eby wrote:

Standard Library Changes/Additions
--

The ``pkgutil`` module should be updated to handle this
specification appropriately, including any necessary changes to
``extend_path()``, ``iter_modules()``, etc.

Specifically the proposed changes and additions to ``pkgutil`` are:

* A new ``extend_virtual_paths(path_entry)`` function, to extend
  existing, already-imported virtual packages' ``__path__`` attributes
  to include any portions found in a new ``sys.path`` entry.  This
  function should be called by applications extending ``sys.path``
  at runtime, e.g. when adding a plugin directory or an egg to the
  path.

  The implementation of this function does a simple top-down traversal
  of ``sys.virtual_packages``, and performs any necessary
  ``get_subpath()`` calls to identify what path entries need to
  be added to each package's ``__path__``, given that `path_entry`
  has been added to ``sys.path``.  (Or, in the case of sub-packages,
  adding a derived subpath entry, based on their parent namespace's
  ``__path__``.) 


When I read about creating __path__ from sys.path, I immediately thought 
of the issue of programs that extend sys.path, and the above is the 
"workaround" for such programs.  but it requires such programs to do 
work, and there are a lot of such programs (I, a relative newbie, have 
had to write some).  As it turns out, I can't think of a situation where 
I have extended sys.path that would result in a problem for fancy 
namespace packages, because so far I've only written modules, not 
packages, and only modules are on the paths that I add to sys.path.  But 
that does not make for a general solution.


Is there some way to create a new __path__ that would reflect the fact 
that it has been dynamically created, rather than set from __init__.py, 
and then when it is referenced, calculate (and cache?) a new value of 
__path__ to actually search?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[Piotr Ożarowski]

+1 (and yay!)
-- 
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl  www.griffith.cc   www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[Nick Coghlan]

On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby  wrote:
> So, without further ado, here it is:

I pushed this version up to the PEPs repo, so it now has a number
(402) and can be read in prettier HTML format:
http://www.python.org/dev/peps/pep-0402/

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream)

[Nick Coghlan]

On Wed, Jul 20, 2011 at 4:53 PM, Kerrick Staley  wrote:
> Nick, can you please apply the patch (will be sent in the following
> email) to the PEP SVN as soon as we get the hard-link issue is figured
> out? Alternatively, could you provide me write access to just the
> pep-0394.txt file so I can update it myself? Thanks.

Done: http://www.python.org/dev/peps/pep-0394/

Main thing needed now is a tracker issue to reference (ideally with a
first cut at a patch) and python-dev's collective blessing to take it
out of Draft status.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream)

[Ned Deily]

In article 
,
 Kerrick Staley  wrote:
> On Mon, Jul 18, 2011 at 3:03 AM, Ned Deily  wrote:
> > I think adding the requirement to mandate hard link vs soft link usage
> > is an unnecessary and unwarranted attempt at optimization.  For
> > instance, IIRC, the OS X installers don't use any hard links: that may
> > complicate the install, plus hard links on OS X HFS* file systems are a
> > bit of a kludge and not necessarily more efficient than symlinks.   It's
> > not a big deal but perhaps the wording should be changed to make a
> > suggestion about hard links vs syminks rather than mandate which should
> > be used.
> 
> Ah, OK. The wording's been changed so that symbolic links will be
> installed on Mac OS X and hard links elsewhere (although maybe
> symbolic links are also better on certain other platforms; I'm not
> sure).
> 
> I do think that specific instructions must be given (rather than just
> a suggestion) because it's indicating what must be done to CPython.
> The instructions *should* be as close as possible to what the
> installer already does, but I'm not entirely sure what the installer
> does by default, and the hard-link recommendation was based off a
> cursory inspection of my own system, so further input from yourself
> and the rest of python-dev would be appreciated.

Thanks for addressing the comments.  Unfortunately, I was relying too 
much on memory and I should have checked more carefully.  I see now 
that, within the framework bin directory for Python 3, the python3 -> 
python3.x link is, in fact, a hard link because it is being produced by 
the standard Makefile target "bininstall".  I was thinking more of the 
optional /usr/local/bin links which are all symlinks.  So, if the Python 
3 installer does is, there's no reason why Python 2 can't do it, I 
suppose.

But if you look at the Python 3 "bininstall" target (Makefile.pre.in 
starting around line 870 or so), you'll see that, for Python 3, symlinks 
are made for all the versioned files except "python3".  I'm not sure 
that there is a particular reason why a distinction was made;  IIRC, the 
other versioned links were added later in the cycle.  The other Python 3 
versioned links could probably be changed to hard links as well.  In the 
end, I don't think it makes a lot of difference.  But it would be better 
if Python 2 and Python 3 were consistent here.

My apologies for the confusion and extra work.

-- 
 Ned Deily,
 n...@acm.org

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise)

[Paul Moore]

On 20 July 2011 03:21, Terry Reedy  wrote:
> Suppose for Windows there were one '.../python' directory wherever the user
> first asks it to be put and that all pythons, not just cpython, are
> installed in directories below that and that the small startup file is
> copied into or linked from the python directory. Then the one python
> directory could be put on the path and left there and never removed by any
> python de-installer (unless perhaps it check that there are no subdirs and
> *asks* the user.

Hmm. Suppose that directory was "C:\Program Files\Python Launcher" (or
"C:\Windows\system32" if you don't want to add an extra directory to
PATH). And suppose that instead of having a startup file per Python
installation you have a single file called py.exe. Then you have the
launcher!

Plus, the launcher has its own uninstaller, making it a "normal" part
of the Windows environment, rather than being a directory created by
something which doesn't get uninstalled.

Plus, the launcher has a means of dealing with the "generic" python,
python2 and python3 commands, which your proposal doesn't.

Plus, the launcher deals with existing versions of Python, which your
proposal doesn't (except by manual intervention).

But yes, the idea is sound, which is why it's so similar to what Vinay
did with the launcher IMO.

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com