Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-30 Thread Sandro Tosi
Hi, sorry for nitpicking, but... On Wed, Jul 20, 2011 at 05:58, P.J. Eby p...@telecommunity.com wrote: ... For those implementing PEP \302 importer objects: the '\' should be removed, right? Cheers, -- Sandro Tosi (aka morph, morpheus, matrixhasu) My website: http://matrixhasu.altervista.org/

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-30 Thread Éric Araujo
Hi Sandro, On Wed, Jul 20, 2011 at 05:58, P.J. Eby p...@telecommunity.com wrote: For those implementing PEP \302 importer objects: the '\' should be removed, right? No. Philip used backslashes to prevent the HTML conversion to transform each and every instance of “PEP \d+” to a link, which

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-30 Thread Sandro Tosi
On Sat, Jul 30, 2011 at 14:57, Éric Araujo mer...@netwok.org wrote: On Wed, Jul 20, 2011 at 05:58, P.J. Eby p...@telecommunity.com wrote: For those implementing PEP \302 importer objects: the '\' should be removed, right? No.  Philip used backslashes to prevent the HTML conversion to

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-27 Thread Barry Warsaw
First off, kudos to PJE for his work on this PEP. He really had the key insight for this new approach, and did a great job of explaining his vision in a clear way so that I think everybody over on import-sig got it. On Jul 20, 2011, at 08:57 AM, P.J. Eby wrote: At 06:46 PM 7/20/2011 +1000, Nick

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-22 Thread Greg Ewing
Antoine Pitrou wrote: The additional confusion lies in the fact that a module can be shadowed by something which is not a module (a mere global variable). I find it rather baffling. I think we're stuck with that as long as we use the same syntax for importing a submodule and importing a

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-22 Thread Greg Ewing
P.J. Eby wrote: from x import y means import x; y = x.y. It actually means slightly more that that if y is a submodule, in which case it means import x.y; y = x.y. -- Greg ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-21 Thread P.J. Eby
At 11:52 AM 7/21/2011 +1000, Nick Coghlan wrote: Trying to change how packages are identified at the Python level makes PEP 382 sound positively appealing. __path__ needs to stay :) In which case, it should be a list, not a sentinel. ;-) Even better would be for these (and sys.path) to be

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-21 Thread P.J. Eby
At 12:59 PM 7/21/2011 -0700, Reliable Domains wrote: I assume that the implicit extend_virtual_paths() would be smart enough to only do real work if there are virtual packages to do it in, so much of the performance costs (bunch of stats) are bounded by the existence of and number of virtual

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-21 Thread Nick Coghlan
On Thu, Jul 21, 2011 at 11:20 PM, P.J. Eby p...@telecommunity.com wrote: This seems to lean in favor of making a simple reiterable wrapper type for the __path__, that only allows you to take the length and iterate over it.  With an appropriate design, it could actually update itself

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-21 Thread Antoine Pitrou
On Tue, 19 Jul 2011 23:58:55 -0400 P.J. Eby p...@telecommunity.com wrote: Anyway, to make a long story short, we came up with an alternative implementation plan that actually solves some other problems besides the one that PEP 382 sets out to solve, and whose implementation a bit is

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-21 Thread Nick Coghlan
On Fri, Jul 22, 2011 at 9:35 AM, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 19 Jul 2011 23:58:55 -0400 P.J. Eby p...@telecommunity.com wrote: Anyway, to make a long story short, we came up with an alternative implementation plan that actually solves some other problems besides the one

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-21 Thread Antoine Pitrou
Le vendredi 22 juillet 2011 à 09:53 +1000, Nick Coghlan a écrit : On Fri, Jul 22, 2011 at 9:35 AM, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 19 Jul 2011 23:58:55 -0400 P.J. Eby p...@telecommunity.com wrote: Anyway, to make a long story short, we came up with an alternative

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-21 Thread Glenn Linderman
On 7/21/2011 5:00 PM, Antoine Pitrou wrote: Le vendredi 22 juillet 2011 à 09:53 +1000, Nick Coghlan a écrit : On Fri, Jul 22, 2011 at 9:35 AM, Antoine Pitrousolip...@pitrou.net wrote: On Tue, 19 Jul 2011 23:58:55 -0400 P.J. Ebyp...@telecommunity.com wrote: Anyway, to make a long story

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-21 Thread Antoine Pitrou
On Thu, 21 Jul 2011 17:31:04 -0700 Glenn Linderman v+pyt...@g.nevcal.com wrote: If I have (on sys.path), a directory x containing a y.py module, and later (on sys.path), another directory x containing a y.py module, what is from x import y supposed to do? OR If I have (on sys.path), a

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-21 Thread Glenn Linderman
On 7/21/2011 5:38 PM, Antoine Pitrou wrote: However, you can have a x.py file and a x directory *in the same base directory which is present in sys.path*, meaning sys.path can't help disambiguate in this case. Ah yes. It means there has to be one more rule for disambiguation, which Nick

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-21 Thread Nick Coghlan
On Fri, Jul 22, 2011 at 10:00 AM, Antoine Pitrou solip...@pitrou.net wrote: Wouldn't it produce confusing situations like the above example? I don't see how it is any more confusing than any other form of module shadowing. For backwards compatibility reasons, the precedence model will be: 1.

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-21 Thread Nick Coghlan
On Fri, Jul 22, 2011 at 10:53 AM, Glenn Linderman v+pyt...@g.nevcal.com wrote: Ah yes.  It means there has to be one more rule for disambiguation, which Nick supplied.  Your case wasn't clear to me from your first description, however.  As long as there is an ordering, and it is documented, it

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-21 Thread Antoine Pitrou
Le vendredi 22 juillet 2011 à 10:58 +1000, Nick Coghlan a écrit : On Fri, Jul 22, 2011 at 10:00 AM, Antoine Pitrou solip...@pitrou.net wrote: Wouldn't it produce confusing situations like the above example? I don't see how it is any more confusing than any other form of module shadowing.

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-21 Thread P.J. Eby
At 03:04 AM 7/22/2011 +0200, Antoine Pitrou wrote: The additional confusion lies in the fact that a module can be shadowed by something which is not a module (a mere global variable). I find it rather baffling. If you move x.py to x/__init__.py, it does *exactly the same thing* in current

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-21 Thread Nick Coghlan
On Fri, Jul 22, 2011 at 11:04 AM, Antoine Pitrou solip...@pitrou.net wrote: Le vendredi 22 juillet 2011 à 10:58 +1000, Nick Coghlan a écrit : On Fri, Jul 22, 2011 at 10:00 AM, Antoine Pitrou solip...@pitrou.net wrote: Wouldn't it produce confusing situations like the above example? I don't

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread Nick Coghlan
On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby p...@telecommunity.com wrote: So, without further ado, here it is: I pushed this version up to the PEPs repo, so it now has a number (402) and can be read in prettier HTML format: http://www.python.org/dev/peps/pep-0402/ Cheers, Nick. -- Nick Coghlan  

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread Piotr Ożarowski
+1 (and yay!) -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 ___ Python-Dev mailing list

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread Glenn Linderman
On 7/19/2011 8:58 PM, P.J. Eby wrote: Standard Library Changes/Additions -- The ``pkgutil`` module should be updated to handle this specification appropriately, including any necessary changes to ``extend_path()``, ``iter_modules()``, etc. Specifically the

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread P.J. Eby
At 06:46 PM 7/20/2011 +1000, Nick Coghlan wrote: On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby p...@telecommunity.com wrote: So, without further ado, here it is: I pushed this version up to the PEPs repo, so it now has a number (402) and can be read in prettier HTML format:

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread P.J. Eby
At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote: When I read about creating __path__ from sys.path, I immediately thought of the issue of programs that extend sys.path, and the above is the workaround for such programs. but it requires such programs to do work, and there are a lot of such

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread Eric V. Smith
On 07/20/2011 08:57 AM, P.J. Eby wrote: At 06:46 PM 7/20/2011 +1000, Nick Coghlan wrote: On Wed, Jul 20, 2011 at 1:58 PM, P.J. Eby p...@telecommunity.com wrote: So, without further ado, here it is: I pushed this version up to the PEPs repo, so it now has a number (402) and can be read in

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread R. David Murray
On Tue, 19 Jul 2011 23:58:55 -0400, P.J. Eby p...@telecommunity.com wrote: Worse, this is not just a problem for new users: it prevents *anyone* from easily splitting a package into separately-installable components. In Perl terms, it would be as if every possible ``Net::`` module on CPAN

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread Neal Becker
I wonder if this fixes the long-standing issue in OS vendor's distributions. In Fedora, for example, there is both arch-specific and non-arch directories: /usr/lib/python2.7 + /usr/lib64/python2.7, for example. Pure python goes into /usr/lib/python2.7, and code including binaries goes into

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread P.J. Eby
At 10:40 AM 7/20/2011 -0400, Neal Becker wrote: I wonder if this fixes the long-standing issue in OS vendor's distributions. In Fedora, for example, there is both arch-specific and non-arch directories: /usr/lib/python2.7 + /usr/lib64/python2.7, for example. Pure python goes into

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread Jeff Hardy
On Tue, Jul 19, 2011 at 8:58 PM, P.J. Eby p...@telecommunity.com wrote: The biggest likely exception to the above would be when a piece of code tries to check whether some package is installed by importing it.  If this is done *only* by importing a top-level module (i.e., not checking for a

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread Erik
On Wed, Jul 20, 2011 at 11:56 AM, Jeff Hardy jdha...@gmail.com wrote: On Tue, Jul 19, 2011 at 8:58 PM, P.J. Eby p...@telecommunity.com wrote: The biggest likely exception to the above would be when a piece of code tries to check whether some package is installed by importing it.  If this is

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread P.J. Eby
At 08:56 AM 7/20/2011 -0700, Jeff Hardy wrote: On Tue, Jul 19, 2011 at 8:58 PM, P.J. Eby p...@telecommunity.com wrote: The biggest likely exception to the above would be when a piece of code tries to check whether some package is installed by importing it. If this is done *only* by importing

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread P.J. Eby
At 12:37 PM 7/20/2011 -0400, Erik wrote: The best solution I can think of would be to have a way for a module to mark itself as finalized (I'm not sure if that's the best term--just the first that popped into my head). This would prevent its __path__ from being created or extended in any way.

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread Eric Snow
On Wed, Jul 20, 2011 at 11:04 AM, P.J. Eby p...@telecommunity.com wrote: Hm.  Here's another variant that might be easier to implement (even in C), and could offer some other advantages as well. Suppose we replace the sys.virtual_packages set() with a sys.virtual_paths dict(): a dictionary

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread Terry Reedy
On 7/20/2011 1:04 PM, P.J. Eby wrote: This part worries me slightly. Imagine a program as such: datagen.py json/foo.js json/bar.js datagen.py uses the files in json/ to generate sample data for a database. In datagen.py is the following code: try: import json except ImportError: import

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread P.J. Eby
At 01:35 PM 7/20/2011 -0600, Eric Snow wrote: This is a really nice solution. So a virtual package is not imported until a submodule of the virtual package is successfully imported Correct... (except for direct import of pure virtual packages). Not correct. ;-) What we do is avoid

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread Eric Snow
On Wed, Jul 20, 2011 at 2:44 PM, P.J. Eby p...@telecommunity.com wrote: At 01:35 PM 7/20/2011 -0600, Eric Snow wrote: This is a really nice solution.  So a virtual package is not imported until a submodule of the virtual package is successfully imported Correct... (except for direct import

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread Glenn Linderman
On 7/20/2011 6:05 AM, P.J. Eby wrote: At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote: When I read about creating __path__ from sys.path, I immediately thought of the issue of programs that extend sys.path, and the above is the workaround for such programs. but it requires such programs

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread P.J. Eby
At 03:22 PM 7/20/2011 -0600, Eric Snow wrote: On Wed, Jul 20, 2011 at 2:44 PM, P.J. Eby p...@telecommunity.com wrote: So, yeah, actually, that's looking pretty sweet. Basically, we just have to throw a virtual_package_paths dict into the sys module, and do the above along with the

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread P.J. Eby
At 03:09 PM 7/20/2011 -0700, Glenn Linderman wrote: On 7/20/2011 6:05 AM, P.J. Eby wrote: At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote: When I read about creating __path__ from sys.path, I immediately thought of the issue of programs that extend sys.path, and the above is the workaround

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread Glenn Linderman
On 7/20/2011 4:03 PM, P.J. Eby wrote: I'd recommend *always* using it, outside of simple startup code. So that is a burden on every program. Documentation would help, but it certainly makes updating sys.path much more complex -- 3 lines (counting import of pkgutil) instead of one, and the

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread Nick Coghlan
On Thu, Jul 21, 2011 at 9:03 AM, P.J. Eby p...@telecommunity.com wrote: Hm.  Yes, there is a way to do something like that, but it would complicate things a bit.  We'd need to: 1. Leave __path__ off of the modules, and always pull them from sys.virtual_package_paths, and Setting __path__ to

Re: [Python-Dev] Draft PEP: Simplified Package Layout and Partitioning

2011-07-20 Thread Eric Snow
On Wed, Jul 20, 2011 at 7:52 PM, Nick Coghlan ncogh...@gmail.com wrote: Even better would be for these (and sys.path) to be list subclasses that did the right thing under the hood as Glenn suggested. Code that *replaces* rather than modifies these attributes would still potentially break