Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-30 Thread Sandro Tosi
On Sat, Jul 30, 2011 at 14:57, Éric Araujo wrote: >> On Wed, Jul 20, 2011 at 05:58, P.J. Eby wrote: >>> For those implementing PEP \302 importer objects: >> >> the '\' should be removed, right? > > No.  Philip used backslashes to prevent the HTML conversion to transform > each and every instance

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-30 Thread Éric Araujo
Hi Sandro, > On Wed, Jul 20, 2011 at 05:58, P.J. Eby wrote: >> For those implementing PEP \302 importer objects: > > the '\' should be removed, right? No. Philip used backslashes to prevent the HTML conversion to transform each and every instance of “PEP \d+” to a link, which gets annoying aft

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-30 Thread Sandro Tosi
Hi, sorry for nitpicking, but... On Wed, Jul 20, 2011 at 05:58, P.J. Eby wrote: ... > For those implementing PEP \302 importer objects: the '\' should be removed, right? Cheers, -- Sandro Tosi (aka morph, morpheus, matrixhasu) My website: http://matrixhasu.altervista.org/ Me at Debian: http://

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-27 Thread Barry Warsaw
First off, kudos to PJE for his work on this PEP. He really had the key insight for this new approach, and did a great job of explaining his vision in a clear way so that I think everybody over on import-sig "got it". On Jul 20, 2011, at 08:57 AM, P.J. Eby wrote: >At 06:46 PM 7/20/2011 +1000, Ni

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-22 Thread Greg Ewing
P.J. Eby wrote: "from x import y" means "import x; y = x.y". It actually means slightly more that that if y is a submodule, in which case it means "import x.y; y = x.y". -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.or

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-22 Thread Greg Ewing
Antoine Pitrou wrote: The additional confusion lies in the fact that a module can be shadowed by something which is not a module (a mere global variable). I find it rather baffling. I think we're stuck with that as long as we use the same syntax for importing a submodule and importing a non-mo

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-21 Thread Nick Coghlan
On Fri, Jul 22, 2011 at 11:04 AM, Antoine Pitrou wrote: > > Le vendredi 22 juillet 2011 à 10:58 +1000, Nick Coghlan a écrit : >> On Fri, Jul 22, 2011 at 10:00 AM, Antoine Pitrou wrote: >> > Wouldn't it produce confusing situations like the above example? >> >> I don't see how it is any more confu

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-21 Thread P.J. Eby
At 03:04 AM 7/22/2011 +0200, Antoine Pitrou wrote: The additional confusion lies in the fact that a module can be shadowed by something which is not a module (a mere global variable). I find it rather baffling. If you move x.py to x/__init__.py, it does *exactly the same thing* in current ver

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-21 Thread Antoine Pitrou
Le vendredi 22 juillet 2011 à 10:58 +1000, Nick Coghlan a écrit : > On Fri, Jul 22, 2011 at 10:00 AM, Antoine Pitrou wrote: > > Wouldn't it produce confusing situations like the above example? > > I don't see how it is any more confusing than any other form of module > shadowing. The additional

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-21 Thread Nick Coghlan
On Fri, Jul 22, 2011 at 10:53 AM, Glenn Linderman wrote: > Ah yes.  It means there has to be one more rule for disambiguation, which > Nick supplied.  Your case wasn't clear to me from your first description, > however.  As long as there is an ordering, and it is documented, it is not > particular

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-21 Thread Nick Coghlan
On Fri, Jul 22, 2011 at 10:00 AM, Antoine Pitrou wrote: > Wouldn't it produce confusing situations like the above example? I don't see how it is any more confusing than any other form of module shadowing. For backwards compatibility reasons, the precedence model will be: 1. Modules and self-con

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-21 Thread Glenn Linderman
On 7/21/2011 5:38 PM, Antoine Pitrou wrote: However, you can have a "x.py" file and a "x" directory *in the same base directory which is present in sys.path*, meaning sys.path can't help disambiguate in this case. Ah yes. It means there has to be one more rule for disambiguation, which Nick s

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-21 Thread Antoine Pitrou
On Thu, 21 Jul 2011 17:31:04 -0700 Glenn Linderman wrote: > > If I have (on sys.path), a directory "x" containing a "y.py" module, and > later (on sys.path), another directory "x" containing a "y.py" module, > what is "from x import y" supposed to do? > > OR > > If I have (on sys.path), a mod

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-21 Thread Glenn Linderman
On 7/21/2011 5:00 PM, Antoine Pitrou wrote: Le vendredi 22 juillet 2011 à 09:53 +1000, Nick Coghlan a écrit : On Fri, Jul 22, 2011 at 9:35 AM, Antoine Pitrou wrote: On Tue, 19 Jul 2011 23:58:55 -0400 "P.J. Eby" wrote: Anyway, to make a long story short, we came up with an alternative impleme

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-21 Thread Antoine Pitrou
Le vendredi 22 juillet 2011 à 09:53 +1000, Nick Coghlan a écrit : > On Fri, Jul 22, 2011 at 9:35 AM, Antoine Pitrou wrote: > > On Tue, 19 Jul 2011 23:58:55 -0400 > > "P.J. Eby" wrote: > >> > >> Anyway, to make a long story short, we came up with an alternative > >> implementation plan that actual

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-21 Thread Nick Coghlan
On Fri, Jul 22, 2011 at 9:35 AM, Antoine Pitrou wrote: > On Tue, 19 Jul 2011 23:58:55 -0400 > "P.J. Eby" wrote: >> >> Anyway, to make a long story short, we came up with an alternative >> implementation plan that actually solves some other problems besides >> the one that PEP 382 sets out to solv

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-21 Thread Antoine Pitrou
On Tue, 19 Jul 2011 23:58:55 -0400 "P.J. Eby" wrote: > > Anyway, to make a long story short, we came up with an alternative > implementation plan that actually solves some other problems besides > the one that PEP 382 sets out to solve, and whose implementation a > bit is easier to explain. (

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-21 Thread Nick Coghlan
On Thu, Jul 21, 2011 at 11:20 PM, P.J. Eby wrote: > This seems to lean in favor of making a simple reiterable wrapper type for > the __path__, that only allows you to take the length and iterate over it. >  With an appropriate design, it could actually update itself automatically, > given a subnam

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-21 Thread P.J. Eby
At 12:59 PM 7/21/2011 -0700, Reliable Domains wrote: I assume that the implicit extend_virtual_paths() would be smart enough to only do real work if there are virtual packages to do it in, so much of the performance costs (bunch of stats) are bounded by the existence of and number of virtual pa

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-21 Thread P.J. Eby
At 11:52 AM 7/21/2011 +1000, Nick Coghlan wrote: Trying to change how packages are identified at the Python level makes PEP 382 sound positively appealing. __path__ needs to stay :) In which case, it should be a list, not a sentinel. ;-) Even better would be for these (and sys.path) to be l

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread Eric Snow
On Wed, Jul 20, 2011 at 7:52 PM, Nick Coghlan wrote: > Even better would be for these (and sys.path) to be list subclasses > that did the right thing under the hood as Glenn suggested. Code that > *replaces* rather than modifies these attributes would still > potentially break virtual packages, bu

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread Nick Coghlan
On Thu, Jul 21, 2011 at 9:03 AM, P.J. Eby wrote: > Hm.  Yes, there is a way to do something like that, but it would complicate > things a bit.  We'd need to: > > 1. Leave __path__ off of the modules, and always pull them from > sys.virtual_package_paths, and Setting __path__ to a sentinel value (

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread Glenn Linderman
On 7/20/2011 4:03 PM, P.J. Eby wrote: I'd recommend *always* using it, outside of simple startup code. So that is a burden on every program. Documentation would help, but it certainly makes updating sys.path much more complex -- 3 lines (counting import of pkgutil) instead of one, and the co

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread P.J. Eby
At 03:09 PM 7/20/2011 -0700, Glenn Linderman wrote: On 7/20/2011 6:05 AM, P.J. Eby wrote: At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote: When I read about creating __path__ from sys.path, I immediately thought of the issue of programs that extend sys.path, and the above is the "workaround

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread P.J. Eby
At 03:22 PM 7/20/2011 -0600, Eric Snow wrote: On Wed, Jul 20, 2011 at 2:44 PM, P.J. Eby wrote: > > So, yeah, actually, that's looking pretty sweet. Basically, we just have to > throw a virtual_package_paths dict into the sys module, and do the above > along with the get_virtual_path() functio

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread Glenn Linderman
On 7/20/2011 6:05 AM, P.J. Eby wrote: At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote: When I read about creating __path__ from sys.path, I immediately thought of the issue of programs that extend sys.path, and the above is the "workaround" for such programs. but it requires such programs

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread Eric Snow
On Wed, Jul 20, 2011 at 2:44 PM, P.J. Eby wrote: > At 01:35 PM 7/20/2011 -0600, Eric Snow wrote: >> >> This is a really nice solution.  So a virtual package is not imported >> until a submodule of the virtual package is successfully imported > > Correct... > >> (except for direct import of pure vi

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread P.J. Eby
At 01:35 PM 7/20/2011 -0600, Eric Snow wrote: This is a really nice solution. So a virtual package is not imported until a submodule of the virtual package is successfully imported Correct... (except for direct import of pure virtual packages). Not correct. ;-) What we do is avoid creati

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread Terry Reedy
On 7/20/2011 1:04 PM, P.J. Eby wrote: This part worries me slightly. Imagine a program as such: datagen.py json/foo.js json/bar.js datagen.py uses the files in json/ to generate sample data for a database. In datagen.py is the following code: try: import json except ImportError: import simple

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread Eric Snow
On Wed, Jul 20, 2011 at 11:04 AM, P.J. Eby wrote: > Hm.  Here's another variant that might be easier to implement (even in C), > and could offer some other advantages as well. > > Suppose we replace the sys.virtual_packages set() with a sys.virtual_paths > dict(): a dictionary that maps from modul

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread P.J. Eby
At 12:37 PM 7/20/2011 -0400, Erik wrote: The best solution I can think of would be to have a way for a module to mark itself as "finalized" (I'm not sure if that's the best term--just the first that popped into my head). This would prevent its __path__ from being created or extended in any way.

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread P.J. Eby
At 08:56 AM 7/20/2011 -0700, Jeff Hardy wrote: On Tue, Jul 19, 2011 at 8:58 PM, P.J. Eby wrote: > The biggest likely exception to the above would be when a piece of > code tries to check whether some package is installed by importing > it. If this is done *only* by importing a top-level module

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread Erik
On Wed, Jul 20, 2011 at 11:56 AM, Jeff Hardy wrote: > On Tue, Jul 19, 2011 at 8:58 PM, P.J. Eby wrote: >> The biggest likely exception to the above would be when a piece of >> code tries to check whether some package is installed by importing >> it.  If this is done *only* by importing a top-leve

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread Jeff Hardy
On Tue, Jul 19, 2011 at 8:58 PM, P.J. Eby wrote: > The biggest likely exception to the above would be when a piece of > code tries to check whether some package is installed by importing > it.  If this is done *only* by importing a top-level module (i.e., not > checking for a ``__version__`` or so

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread P.J. Eby
At 10:40 AM 7/20/2011 -0400, Neal Becker wrote: I wonder if this fixes the long-standing issue in OS vendor's distributions. In Fedora, for example, there is both arch-specific and non-arch directories: /usr/lib/python2.7 + /usr/lib64/python2.7, for example. Pure python goes into /usr/lib/pyt

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread Neal Becker
I wonder if this fixes the long-standing issue in OS vendor's distributions. In Fedora, for example, there is both arch-specific and non-arch directories: /usr/lib/python2.7 + /usr/lib64/python2.7, for example. Pure python goes into /usr/lib/python2.7, and code including binaries goes into /u

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread R. David Murray
On Tue, 19 Jul 2011 23:58:55 -0400, "P.J. Eby" wrote: > Worse, this is not just a problem for new users: it prevents *anyone* > from easily splitting a package into separately-installable > components. In Perl terms, it would be as if every possible ``Net::`` > module on CPAN had to be bundled u

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread Eric V. Smith
On 07/20/2011 08:57 AM, P.J. Eby wrote: > At 06:46 PM 7/20/2011 +1000, Nick Coghlan wrote: >> On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby wrote: >> > So, without further ado, here it is: >> >> I pushed this version up to the PEPs repo, so it now has a number >> (402) and can be read in prettier HTML

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread P.J. Eby
At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote: When I read about creating __path__ from sys.path, I immediately thought of the issue of programs that extend sys.path, and the above is the "workaround" for such programs. but it requires such programs to do work, and there are a lot of suc

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread P.J. Eby
At 06:46 PM 7/20/2011 +1000, Nick Coghlan wrote: On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby wrote: > So, without further ado, here it is: I pushed this version up to the PEPs repo, so it now has a number (402) and can be read in prettier HTML format: http://www.python.org/dev/peps/pep-0402/ Te

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread Glenn Linderman
On 7/19/2011 8:58 PM, P.J. Eby wrote: Standard Library Changes/Additions -- The ``pkgutil`` module should be updated to handle this specification appropriately, including any necessary changes to ``extend_path()``, ``iter_modules()``, etc. Specifically the propos

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread Piotr Ożarowski
+1 (and yay!) -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 ___ Python-Dev mailing list Python-D

Re: [Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-20 Thread Nick Coghlan
On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby wrote: > So, without further ado, here it is: I pushed this version up to the PEPs repo, so it now has a number (402) and can be read in prettier HTML format: http://www.python.org/dev/peps/pep-0402/ Cheers, Nick. -- Nick Coghlan   |   ncogh...@gmail.c

[Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

2011-07-19 Thread P.J. Eby
So, over on the Import-SIG, we were talking about the implementation and terminology for PEP 382, and it became increasingly obvious that things were, well, not entirely okay in the "implementation is easy to explain" department. Anyway, to make a long story short, we came up with an alternati