Re: [Python-Dev] Dropping __init__.py requirement for subpackages
+1 Excellent Change +1 Minimal Backward Compatibility Difficulties I think this would also help quite a bit with newbie adoption of Python. I've had to explain this un-feature on numerous occassions and it given how smart Python is, I've wondered why it has this requirement. If you look in various open source packages, you'll find that 95% of these __init__.py files are empty. The ones at my work actually say: # stupid Python requirement, don't remove this file Why? Someone decided to remove files of length 0 in our repository without realizing the consequences. Since it had the __init__.pyc file around, it still worked... till one brought down a fresh copy of the repository and then it just stopped working. Quite a bit of hair pulling that one caused us. The only case where this might cause a problem is with resource directories that only contain .html, .jpg and other files. So, perhpas this feature would only turn a directory into a package if it didn't have any .py files. It could also trigger only when the package is explicitly imported? Good luck /w the pitch-fork wielding users and telling the old-timers where they can keep their backward compatibility. Clark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Thomas Wouters wrote: On 4/27/06, Thomas Wouters [EMAIL PROTECTED] wrote: Alrighty then. The list has about 12 hours to convince me (and you) that it's a bad idea to generate that warning. I'll be asleep by the time the trunk un-freezes, and I have a string of early meetings tomorrow. I'll get to it somewhere in the afternoon :) I could check it in, except the make-testall I ran overnight showed a small problem: the patch would generate a number of spurious warnings in the trunk: /home/thomas/python/python/trunk/Lib/gzip.py:9: ImportWarning: Not importing directory '/home/thomas/python/python/trunk/Modules/zlib': missing __init__.py /home/thomas/python/python/trunk/Lib/ctypes/__init__.py:8: ImportWarning: Not importing directory '/home/thomas/python/python/trunk/Modules/_ctypes': missing __init__.py (and a few more zlib ones.) The reason for that is that ./Modules is added to the import path, by a non-installed Python. This is because of the pre-distutils Modules/Setup-style build method of modules (which is still sometimes used.) I can't find where Modules is added to sys.path, though, even if I wanted to remove it :) So, do we: a) forget about the warning because of the layout of the svn tree (bad, imho) 2) rename Modules/zlib and Modules/_ctypes to avoid the warning (inconvenient, but I don't know how inconvenient) - fix the build procedure so Modules isn't added to sys.path unless it absolutely has to (which is only very rarely the case, I believe) or lastly, make regrtest.py ignore those specific warnings? Would not another way be to make sure Modules is moved *behind* the setup.py build directory on sys.path? Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Thomas Wouters wrote: Indeed! I hadn't realized that, although I might've if I'd been able to find where Modules is put on sys.path. And, likewise, I would do as you suggest (which feels like the right thing) if I could only find out where Modules is put on sys.path :) I don't have time to search for it today nor, probably, this weekend (which is a party weekend in the Netherlands.) I'll get to it eventually, although a helpful hint from an old-timer who remembers as far back as Modules/Setup would be welcome. :) With some debugging, I found it out: search_for_exec_prefix looks for the presence of Modules/Setup; if that is found, it strips off /Setup, leaving search_for_exec_prefix with -1. This then gets added to sys.path with /* Finally, on goes the directory for dynamic-load modules */ strcat(buf, exec_prefix); I wasn't following exactly, so I might have mixed something up, but... it appears that this problem here comes from site.py adding the build directory for the distutils dynamic objects even after Modules. The site.py code is # XXX This should not be part of site.py, since it is needed even when # using the -S option for Python. See http://www.python.org/sf/586680 def addbuilddir(): Append ./build/lib.platform in case we're running in the build dir (especially for Guido :-) from distutils.util import get_platform s = build/lib.%s-%.3s % (get_platform(), sys.version) s = os.path.join(os.path.dirname(sys.path[-1]), s) sys.path.append(s) I would suggest fixing #586680: Add build/lib.* before adding dynamic-load modules, by moving the code into Modules/getpath.c. You should be able to use efound 0 as an indication that this is indeed the build directory. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/27/06, Guido van Rossum [EMAIL PROTECTED] wrote: On 4/26/06, Thomas Wouters [EMAIL PROTECTED] wrote: Of course, I only consider *my* reasons to be valid, and mine weren't knee-jerk or tool-related. I don't think Python should be going Oh, what you wanted wasn't possible, but I think I know what you wanted, let me do it for you, first of all because it's not very Pythonic, and second of all because it doesn't lower the learning curve, it just delays some upward motion a little (meaning the curve may become steeper, later.) A clear warning, on the other hand, can be a helpful nudge towards the 'a-HA' moment. That still sounds like old-timer reasoning. Long ago we were very close to defining a package as a directory -- with none of this must contain __init__.py or another *.py file nonsense. IIRC the decision to make __init__.py mandatory faced opposition too, since people were already doing packages with just directories (which is quite clean and elegant, and that's also how it was in Java), but I added it after seeing a few newbies tear out their hair. OK. After due consideration, I'm happy to accept the change. (Call that +0, it doesn't bother me much personally either way). Although reading the above paragraph, I get the impression that you are saying that __init__.py was originally added to help newbies, and yet you are now saying the exact opposite. I'll leave you to resolve that particular contradiction, though... FWIW, I still have every confidence in your judgement about features. However, I'd have to say that your timing sucks :-) Your initial message read to me as Quick! I'm about to get lynched here - can I have the OK to shove this change in before a2 goes out? (OK, 2.5 isn't feature frozen until a3, so maybe you only meant a3, but you clearly wanted a quick response). So it's hard to expect anything other than immediate knee-jerk responses. And those are usually driven by personal experience (implying less consideration of newbie mistakes from this type of audience) and unfocused fear of breakage. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/27/06, Paul Moore [EMAIL PROTECTED] wrote: However, I'd have to say that your timing sucks :-) Your initial message read to me as Quick! I'm about to get lynched here - can I have the OK to shove this change in before a2 goes out? And this just proves that my response wasn't anywhere near as considered as I thought. You explicitly said 2.6, which I'd forgotten. I have no problem with going with your instinct in 2.6. But I do believe that there may be more breakage than you have considered (PEP 302 experience talking here :-)) so get it in early rather than late! Of course this is all a bit moot now, as the decision seems to have been made. Sigh. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On Thursday 27 April 2006 17:47, Paul Moore wrote: FWIW, I still have every confidence in your judgement about features. However, I'd have to say that your timing sucks :-) Your initial message read to me as Quick! I'm about to get lynched here - can I have the OK to shove this change in before a2 goes out? (OK, 2.5 isn't feature frozen until a3, so maybe you only meant a3, but you clearly wanted a quick response). Feature freeze is the day we release beta1. New features go in until then - after that, not without approval and a general concensus that the changes have a substantial cost/benefit for breaking the feature freeze. Or if Guido gets Google developers parading him in effigy around the office and needs to get them off his back. wink Anthony -- Anthony Baxter [EMAIL PROTECTED] It's never too late to have a happy childhood. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum guido at python.org writes: The requirement that a directlry must contain an __init__.py file before it is considered a valid package has always been controversial. It's designed to prevent the existence of a directory with a common name like time or string from preventing fundamental imports to work. But the feature often trips newbie Python programmers (of which there are a lot at Google, at our current growth rate we're probably training more new Python programmers than any other organization worldwide . Might I suggest an alternative? I too find it cumbersome to have to litter my directory tree with __init__.py iles. However, rather than making modules be implicit (Explicit is better than implicit), I would rather see a more powerful syntax for declaring modules. Specifically, what I would like to see is a way for a top-level __init__.py to explicitly list which subdirectories are modules. Rather than spreading that information over a bunch of different __init__.py files, I would much rather have the information be centralized in a single text file for the whole package. Just as we have an __all__ variable that indicates which symbols are to be exported, we could have a '__submodules__' array which explicitly calls out the list of submodule directory names. Or perhaps more simply, just have some code in the top-level __init__.py that creates (but does not load) the module objects for the various sub-modules. The presence of __init__.py could perhaps also indicate the root of a *standalone* module tree; submodules that don't have their own __init__.py, but which are declared indirectly via an ancestor are assumed by convention to be 'component' modules which are not intended to operate outside of their parent. (In other words, the presence of the __init__.py signals that that tree is separately exportable.) I'm sure that someone who is familiar with the import machinery could whip up something like this in a matter of minutes. As far as the compatibility with tools argument goes, I say, break em :) Those tool programmers need a hobby anyway :) -- Talin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. I'm going to skip the discussion thread (or is it a flame war? cannot tell from the thread pattern), but here are my votes: +0 on dropping the requirement for subpackages in 2.X (this will break tools, but probably not break much code, and fixing the tools should be straightforward) (+1 on dropping it in 3.X) -1 on dropping the requirement for packages in 2.X (-0 on dropping it in 3.X) +0 on making this change in 2.5 /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. I haven't scanned this whole thread yet, but my first thought was to just try to find a way to give a better error message if we find a candidate package directory, but there's no __init__.py file. i.e. something like: ImportError: __init__.py not found in directory 'whatever/foo' for package 'foo' I like the fact that __init__.py documents, right there in the file system directory listing, that the current directory is a Python package. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido One particular egregious problem is that *subpackage* are subject Guido to the same rule. It so happens that there is essentially only Guido one top-level package in the Google code base, and it already has Guido an __init__.py file. But developers create new subpackages at a Guido frightening rate, and forgetting to do touch __init__.py has Guido caused many hours of lost work, not to mention injuries due to Guido heads banging against walls. That's why God created make: install: touch __init__.py blah blah blah Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Not that it would count in any way, but I'd prefer to keep it. How would I mark a subdirectory as not-a-package otherwise? Guido What's the use case for that? Have you run into this requirement? Yes, we run into it. We typically install a package with any resources in a resources subdirectory. Now, those resources subdirectories generally don't contain Python files (Glade files are the most frequent occupants), but there's no reason they couldn't contain Python files. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Guido van Rossum [EMAIL PROTECTED] wrote:[...] So I have a very simple proposal: keep the __init__.py requirement fortop-level pacakages, but drop it for subpackages. This should be asmall change. I'm hesitant to propose *anything* new for Python 2.5,so I'm proposing it for 2.6; if Neal and Anthony think this would beokay to add to 2.5, they can do so. Damn these threads are so quick they are born and die off in 24 hours and don't give enough time for people to comment :( I'm a bit late in the thread, but I'm +1 (for 2.5) for the following reason. There are mainly two use cases for having a couple of modules foo.bar, and foo.zbr: 1. foo.bar and foo.zbr are both part of the same _package_, and are distributed together; 2. foo.bar and foo.zbr are two independent modules, distributed separately, but which share a common 'foo' _namespace_, to denote affiliation with a project. The use case #1 is arguably more common, but use case #2 is also very relevant. It happens for a lot of GNOME python bindings, for example, where we used to have gnome, gnome.ui, gnome.vfs, gnome.applet, etc. Now the problem. Suppose you have the source package python-foo-bar, which installs $pythondir/foo/__init__.py and $pythondir/foo/bar.py. This would make a module called foo.bar available. Likewise, you can have the source package python-foo-zbr, which installs $pythondir/foo/__init__.py and $pythondir/foo/zbr.py. This would make a module called foo.zbr available. The two packages above install the file $pythondir/foo/__init__.py. If one of them adds some content to __init__.py, the other one will overwrite it. Packaging these two packages for e.g. debian would be extremely difficult, because no two .deb packages are allowed to intall the same file. One solution is to generate the __init__.py file with post-install hooks and shell scripts. Another solution would be for example to have only python-foo-bar install the __init__.py file, but then python-foo-zbr would have to depend on python-foo-bar, while they're not really related. I hope I made the problem clear enough, and I hope people find this a compelling argument in favour of eliminating the need for __init__.py. Regards,Gustavo Carneiro. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Gustavo Carneiro [EMAIL PROTECTED] writes: Now the problem. Suppose you have the source package python-foo-bar, which installs $pythondir/foo/__init__.py and $pythondir/foo/bar.py. This would make a module called foo.bar available. Likewise, you can have the source package python-foo-zbr, which installs $pythondir/foo/__init__.py and $pythondir/foo/zbr.py. This would make a module called foo.zbr available. The two packages above install the file $pythondir/foo/__init__.py. If one of them adds some content to __init__.py, the other one will overwrite it. Packaging these two packages for e.g. debian would be extremely difficult, because no two .deb packages are allowed to intall the same file. One solution is to generate the __init__.py file with post-install hooks and shell scripts. Another solution would be for example to have only python-foo-bar install the __init__.py file, but then python-foo-zbr would have to depend on python-foo-bar, while they're not really related. Yet another solution would be to put foo/__init__.py into a third package, e.g. python-foo, on which both python-foo-bar and python-foo-zbr depend. Bernhard -- Intevation GmbH http://intevation.de/ Skencil http://skencil.org/ Thuban http://thuban.intevation.org/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/27/06, Thomas Wouters [EMAIL PROTECTED] wrote: Alrighty then. The list has about 12 hours to convince me (and you) that it's a bad idea to generate that warning. I'll be asleep by the time the trunk un-freezes, and I have a string of early meetings tomorrow. I'll get to it somewhere in the afternoon :) I could check it in, except the make-testall I ran overnight showed a small problem: the patch would generate a number of spurious warnings in the trunk:/home/thomas/python/python/trunk/Lib/gzip.py:9: ImportWarning: Not importing directory '/home/thomas/python/python/trunk/Modules/zlib': missing __init__.py /home/thomas/python/python/trunk/Lib/ctypes/__init__.py:8: ImportWarning: Not importing directory '/home/thomas/python/python/trunk/Modules/_ctypes': missing __init__.py(and a few more zlib ones.) The reason for that is that ./Modules is added to the import path, by a non-installed Python. This is because of the pre-distutils Modules/Setup-style build method of modules (which is still sometimes used.) I can't find where Modules is added to sys.path, though, even if I wanted to remove it :)So, do we:a) forget about the warning because of the layout of the svn tree (bad, imho)2) rename Modules/zlib and Modules/_ctypes to avoid the warning (inconvenient, but I don't know how inconvenient) - fix the build procedure so Modules isn't added to sys.path unless it absolutely has to (which is only very rarely the case, I believe)or lastly, make regrtest.py ignore those specific warnings? -- Thomas Wouters [EMAIL PROTECTED]Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 03:48 PM 4/27/2006 +0200, Bernhard Herzog wrote: Gustavo Carneiro [EMAIL PROTECTED] writes: Now the problem. Suppose you have the source package python-foo-bar, which installs $pythondir/foo/__init__.py and $pythondir/foo/bar.py. This would make a module called foo.bar available. Likewise, you can have the source package python-foo-zbr, which installs $pythondir/foo/__init__.py and $pythondir/foo/zbr.py. This would make a module called foo.zbr available. The two packages above install the file $pythondir/foo/__init__.py. If one of them adds some content to __init__.py, the other one will overwrite it. Packaging these two packages for e.g. debian would be extremely difficult, because no two .deb packages are allowed to intall the same file. One solution is to generate the __init__.py file with post-install hooks and shell scripts. Another solution would be for example to have only python-foo-bar install the __init__.py file, but then python-foo-zbr would have to depend on python-foo-bar, while they're not really related. Yet another solution would be to put foo/__init__.py into a third package, e.g. python-foo, on which both python-foo-bar and python-foo-zbr depend. Or you can package them with setuptools, and declare foo to be a namespace package. If installing in the mode used for building RPMs and debs, there will be no __init__.py. Instead, each installs a .pth file that ensures a dummy package object is created in sys.modules with an appropriate __path__. This solution is packaging-system agnostic and doesn't require any special support from the packaging tool. (The downside, however, is that neither foo.bar nor foo.zbr's __init__.py will be allowed to have any content, since in some installation scenarios there will be no __init__.py at all.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/27/06, Phillip J. Eby [EMAIL PROTECTED] wrote: At 03:48 PM 4/27/2006 +0200, Bernhard Herzog wrote:Gustavo Carneiro [EMAIL PROTECTED] writes: Now the problem.Suppose you have the source package python-foo-bar, which installs $pythondir/foo/__init__.py and $pythondir/foo/bar.py.This would make a module called foo.bar available.Likewise, you can have the source package python-foo-zbr, which installs $pythondir/foo/__init__.py and $pythondir/foo/zbr.py.This would make a module called foo.zbr available. The two packages above install the file $pythondir/foo/__init__.py.If one of them adds some content to __init__.py, the other one will overwrite it.Packaging these two packages for e.g. debian would be extremely difficult, because no two .deb packages are allowed to intall the same file. One solution is to generate the __init__.py file with post-install hooks and shell scripts.Another solution would be for example to have only python-foo-bar install the __init__.py file, but then python-foo-zbr would have to depend on python-foo-bar, while they're not really related.Yet another solution would be to put foo/__init__.py into a thirdpackage, e.g. python-foo, on which both python-foo-bar and python-foo-zbr depend. You can't be serious. One package just to install a __init__.py file? Or you can package them with setuptools, and declare foo to be a namespacepackage. Let's not assume setuptools are always used, shall we? If installing in the mode used for building RPMs and debs, therewill be no __init__.py.Instead, each installs a .pth file that ensures adummy package object is created in sys.modules with an appropriate__path__.This solution is packaging-system agnostic and doesn't require any special support from the packaging tool. I don't understand this solution. How can a .pth file create a 'dummy package'? Remember that the objective is to have foo.bar and foo.zbr modules, not just bar and zbr. But in any case, it already sounds like a convoluted solution. No way it can beat the obvious/simple solution: to remove the need to have __init__.py in the first place. (The downside, however, is that neither foo.bar nor foo.zbr's __init__.pywill be allowed to have any content, since in some installation scenarios there will be no __init__.py at all.) That's ok in the context of this proposal (not having __init__.py at all). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/27/06, Gustavo Carneiro [EMAIL PROTECTED] wrote: On 4/27/06, Phillip J. Eby [EMAIL PROTECTED] wrote: At 03:48 PM 4/27/2006 +0200, Bernhard Herzog wrote:Gustavo Carneiro [EMAIL PROTECTED] writes: Now the problem.Suppose you have the source package python-foo-bar, which installs $pythondir/foo/__init__.py and $pythondir/foo/bar.py.This would make a module called foo.bar available.Likewise, you can have the source package python-foo-zbr, which installs $pythondir/foo/__init__.py and $pythondir/foo/zbr.py.This would make a module called foo.zbr available. The two packages above install the file $pythondir/foo/__init__.py.If one of them adds some content to __init__.py, the other one will overwrite it.Packaging these two packages for e.g. debian would be extremely difficult, because no two .deb packages are allowed to intall the same file. Yet another solution would be to put foo/__init__.py into a thirdpackage, e.g. python-foo, on which both python-foo-bar and python-foo-zbr depend. You can't be serious. One package just to install a __init__.py file?Sure. Have you counted the number of 'empty' packages in Debian lately? Besides, Guido's original proposal is not a fix for your problem, either; he only proposes to change the requirement for *sub*packages. -- Thomas Wouters [EMAIL PROTECTED]Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/27/06, Thomas Wouters [EMAIL PROTECTED] wrote: On 4/27/06, Gustavo Carneiro [EMAIL PROTECTED] wrote: On 4/27/06, Phillip J. Eby [EMAIL PROTECTED] wrote: At 03:48 PM 4/27/2006 +0200, Bernhard Herzog wrote:Gustavo Carneiro [EMAIL PROTECTED] writes: Now the problem.Suppose you have the source package python-foo-bar, which installs $pythondir/foo/__init__.py and $pythondir/foo/bar.py.This would make a module called foo.bar available.Likewise, you can have the source package python-foo-zbr, which installs $pythondir/foo/__init__.py and $pythondir/foo/zbr.py.This would make a module called foo.zbr available. The two packages above install the file $pythondir/foo/__init__.py.If one of them adds some content to __init__.py, the other one will overwrite it.Packaging these two packages for e.g. debian would be extremely difficult, because no two .deb packages are allowed to intall the same file. Yet another solution would be to put foo/__init__.py into a thirdpackage, e.g. python-foo, on which both python-foo-bar and python-foo-zbr depend. You can't be serious. One package just to install a __init__.py file? Sure. Have you counted the number of 'empty' packages in Debian lately? Sure. That is already a problem; let's not make it a worse problem for no good reason; I haven't heard a good reason to keep the __init__.py file besides backward compatibility concerns. Besides, Guido's original proposal is not a fix for your problem, either; he only proposes to change the requirement for *sub*packages. It *is* a solution for my problem. I don't need the __init__.py file for anything, since I don't need anything defined in the the 'foo' namespace, only the subpackages foo.bar and foo.zbr . ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/27/06, Gustavo Carneiro [EMAIL PROTECTED] wrote: Besides, Guido's original proposal is not a fix for your problem, either; he only proposes to change the requirement for *sub*packages. It *is* a solution for my problem. I don't need the __init__.py file for anything, since I don't need anything defined in the the 'foo' namespace, only the subpackages foo.bar and foo.zbr No. Guido's original proposal is not a fix for your problem, because *it doesn't affect the 'foo' namespace*. Guido's original proposal still requires foo/__init__.py for your namespace to work, it just makes foo/bar/__init__.py and foo/zbr/__init__.py optional. -- Thomas Wouters [EMAIL PROTECTED]Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/27/06, Thomas Wouters [EMAIL PROTECTED] wrote: On 4/27/06, Gustavo Carneiro [EMAIL PROTECTED] wrote: Besides, Guido's original proposal is not a fix for your problem, either; he only proposes to change the requirement for *sub*packages. It *is* a solution for my problem. I don't need the __init__.py file for anything, since I don't need anything defined in the the 'foo' namespace, only the subpackages foo.bar and foo.zbr No. Guido's original proposal is not a fix for your problem, because *it doesn't affect the 'foo' namespace*. Guido's original proposal still requires foo/__init__.py for your namespace to work, it just makes foo/bar/__init__.py and foo/zbr/__init__.py optional. Damn, you're right, I confused subpackage with submodule :P In that case, can I counter-propose to stop requiring the __init__.py file in [foo/__init__.py, foo/bar.py] ? ;-) The proposal would mean that a directory 'foo' with a single file bar.py would make the module 'foo.bar' available if the parent directory of 'foo' is in sys.path./me faces the pitchforks. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/27/06, Thomas Wouters [EMAIL PROTECTED] wrote: I could check it in, except the make-testall I ran overnight showed a small problem: the patch would generate a number of spurious warnings in the trunk: /home/thomas/python/python/trunk/Lib/gzip.py:9: ImportWarning: Not importing directory '/home/thomas/python/python/trunk/Modules/zlib': missing __init__.py /home/thomas/python/python/trunk/Lib/ctypes/__init__.py:8: ImportWarning: Not importing directory '/home/thomas/python/python/trunk/Modules/_ctypes': missing __init__.py (and a few more zlib ones.) The reason for that is that ./Modules is added to the import path, by a non-installed Python. This is because of the pre-distutils Modules/Setup-style build method of modules (which is still sometimes used.) I can't find where Modules is added to sys.path, though, even if I wanted to remove it :) So, do we: a) forget about the warning because of the layout of the svn tree (bad, imho) 2) rename Modules/zlib and Modules/_ctypes to avoid the warning (inconvenient, but I don't know how inconvenient) - fix the build procedure so Modules isn't added to sys.path unless it absolutely has to (which is only very rarely the case, I believe) or lastly, make regrtest.py ignore those specific warnings? I'd say the latter. That's how we deal with warnings during the test suite in general don't we? -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Dropping __init__.py requirement for subpackages
(Context: There's a large crowd with pitchforks and other sharp pointy farm implements just outside the door of my office at Google. They are making an unbelievable racket. It appears they are Google engineers who have been bitten by a misfeature of Python, and they won't let me go home before I have posted this message.) The requirement that a directlry must contain an __init__.py file before it is considered a valid package has always been controversial. It's designed to prevent the existence of a directory with a common name like time or string from preventing fundamental imports to work. But the feature often trips newbie Python programmers (of which there are a lot at Google, at our current growth rate we're probably training more new Python programmers than any other organization worldwide :-). One particular egregious problem is that *subpackage* are subject to the same rule. It so happens that there is essentially only one top-level package in the Google code base, and it already has an __init__.py file. But developers create new subpackages at a frightening rate, and forgetting to do touch __init__.py has caused many hours of lost work, not to mention injuries due to heads banging against walls. So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
* Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. Not that it would count in any way, but I'd prefer to keep it. How would I mark a subdirectory as not-a-package otherwise? echo raise ImportError __init__.py ? nd -- Das, was ich nicht kenne, spielt stückzahlmäßig *keine* Rolle. -- Helmut Schellong in dclc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. So this would mean that current non-package subdirectories in a package (that contain things like data files or configuration info) would become packages with no modules in them? sharpening-my-farm-implements-ly y'rs, -- Benji York ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, André Malo [EMAIL PROTECTED] wrote: * Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. Not that it would count in any way, but I'd prefer to keep it. How would I mark a subdirectory as not-a-package otherwise? What's the use case for that? Have you run into this requirement? And even if you did, was there a requirement that the subdirectory's name be the same as a standard library module? If the subdirectory's name is not constrained, the easiest way to mark it as a non-package is to put a hyphen or dot in its name; if you can't do that, at least name it something that you don't need to import. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Benji York [EMAIL PROTECTED] wrote: Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. So this would mean that current non-package subdirectories in a package (that contain things like data files or configuration info) would become packages with no modules in them? Yup. Of course unless you try to import from them that wouldn't particularly hurt, except if the subdir name happens to be the same as a module name. Note that absolute import (which will be turned on for all in 2.6) will solve the ambiguity; the only ambiguity left would be if you had a module foo.py and also a non-package subdirectory foo. But that's just asking for trouble. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Barry Warsaw [EMAIL PROTECTED] wrote: On Wed, 2006-04-26 at 10:16 -0700, Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. But if it's there, then nothing changes, right? IOW, if we want to expose names in the subpackage's namespace, we can still do it in the subpackage's __init__.py. It's just that otherwise empty subpackage __init__.py files won't be required. Correct. This is an important clarification. What would the following print? import toplevel.sub1.sub2 print toplevel.sub1.sub2.__file__ If it's path/sub1/sub2 then that kind of breaks a common idiom of using os.path.dirname() on a module's __file__ to find co-located resources. Or at least, you have to know whether to dirname its __file__ or not (which might not be too bad, since you'd probably know how that package dir is organized anyway). Oh, cool gray area. I propose that if there's no __init__.py it prints 'path/sub1/sun2/' i.e. with a trailing slash; that causes dirname to just strip the '/'. (It would be a backslash on Windows of course). I dunno. Occasionally it trips me up too, but it's such an obvious and easy fix that it's never bothered me enough to care. But you're not a newbie. for a newbie who's just learned about packages, is familiar with Java, and missed one line in the docs, it's an easy mistake to make and a tough one to debug. I can't think of an example, but I suppose it's still possible that lifting this requirement could cause some in-package located data directories to be mistakenly importable. I'd be somewhat more worried about frameworks that dynamically import things having to be more cautious about cleansing their __import__() arguments now. But (assuming 2.6 and absolute import) what would be the danger of importing such a package? Presumably it contains no *.py or *.so files so there's no code there; but even so you'd have to go out of your way to import it (since if the directory exists it can't also be a subpackage or submodule name that's in actual use). I'd be -1 but the remote possibility of you being burned at the stake by your fellow Googlers makes me -0 :). I'm not sure I understand what your worry is. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
* Guido van Rossum wrote: On 4/26/06, André Malo [EMAIL PROTECTED] wrote: * Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. Not that it would count in any way, but I'd prefer to keep it. How would I mark a subdirectory as not-a-package otherwise? What's the use case for that? Have you run into this requirement? And even if you did, was there a requirement that the subdirectory's name be the same as a standard library module? If the subdirectory's name is not constrained, the easiest way to mark it as a non-package is to put a hyphen or dot in its name; if you can't do that, at least name it something that you don't need to import. Actually I have no problems with the change from inside python, but from the POV of tools, which walk through the directories, collecting/separating python packages and/or supplemental data directories. It's an explicit vs. implicit issue, where implicit would mean kind of heuristics from now on. IMHO it's going to break existing stuff [1] and should at least not be done in such a rush. nd [1] Well, it does break some of mine ;-) -- Da fällt mir ein, wieso gibt es eigentlich in Unicode kein i mit einem Herzchen als Tüpfelchen? Das wär sooo süüss! -- Björn Höhrmann in darw ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 10:16 AM 4/26/2006 -0700, Guido van Rossum wrote:
So I have a very simple proposal: keep the __init__.py requirement for
top-level pacakages, but drop it for subpackages.
Note that many tools exist which have grown to rely on the presence of
__init__ modules. Also, although your proposal would allow imports to work
reasonably well, tools that are actively looking for packages would need to
have some way to distinguish package directories from others.
My counter-proposal: to be considered a package, a directory must contain
at least one module (which of course can be __init__). This allows the is
it a package? question to be answered with only one directory read, as is
the case now. Think of it also as a nudge in favor of flat is better than
nested.
This tweak would also make it usable for top-level directories, since the
mere presence of a 'time' directory wouldn't get in the way of anything.
The thing more likely to have potential for problems is that many Python
projects have a test directory that isn't intended to be a package, and
thus may interfere with imports from the stdlib 'test' package. Whether
this is really a problem or not, I don't know.
But, we could treat packages without __init__ as namespace packages. That
is, set their __path__ to encompass similarly-named directories already on
sys.path, so that the init-less package doesn't interfere with other
packages that have the same name.
This would require a bit of expansion to PEP 302, but probably not
much. Most of the rest is existing technology, and we've already begun
migrating stdlib modules away from doing their own hunting for __init__ and
other files, towards using the pkgutil API.
By the way, one small precedent for packages without __init__: setuptools
generates such packages using .pth files when a package is split between
different distributions but are being installed by a system packaging
tool. In such cases, *both* parts of the package can't include an
__init__, because the packaging tool (e.g. RPM) is going to complain that
the shared file is a conflict. So setuptools generates a .pth file that
creates a module object with the right name and initializes its __path__ to
point to the __init__-less directory.
This should be a small change.
Famous last words. :) There's a bunch of tools that it's not going to
work properly with, and not just in today's stdlib. (Think documentation
tools, distutils extensions, IDEs...)
Are you sure you wouldn't rather just write a GoogleImporter class to fix
this problem? Append it to sys.path_hooks, clear sys.path_importer_cache,
and you're all set. For that matter, if you have only one top-level
package, put the class and the installation code in that top-level
__init__, and you're set to go.
And that approach will work with Python back to version 2.3; no waiting for
an upgrade (unless Google is still using 2.2, of course).
Let's see, the code would look something like:
class GoogleImporter:
def __init__(self, path):
if not os.path.isdir(path):
raise ImportError(Not for me)
self.path = os.path.realpath(path)
def find_module(self, fullname, path=None):
# Note: we ignore 'path' argument since it is only used via
meta_path
subname = fullname.split(.)[-1]
if os.path.isdir(os.path.join(self.path, subname)):
return self
path = [self.path]
try:
file, filename, etc = imp.find_module(subname, path)
except ImportError:
return None
return ImpLoader(fullname, file, filename, etc)
def load_module(self, fullname):
import sys, new
subname = fullname.split(.)[-1]
path = os.path.join(self.path, subname)
module = sys.modules.setdefault(fullname, new.module(fullname))
module.__dict__.setdefault('__path__',[]).append(path)
return module
class ImpLoader:
def __init__(self, fullname, file, filename, etc):
self.file = file
self.filename = filename
self.fullname = fullname
self.etc = etc
def load_module(self, fullname):
try:
mod = imp.load_module(fullname, self.file, self.filename,
self.etc)
finally:
if self.file:
self.file.close()
return mod
import sys
sys.path_hooks.append(GoogleImporter)
sys.path_importer_cache.clear()
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 02:07 PM 4/26/2006 -0400, Phillip J. Eby wrote: def find_module(self, fullname, path=None): # Note: we ignore 'path' argument since it is only used via meta_path subname = fullname.split(.)[-1] if os.path.isdir(os.path.join(self.path, subname)): return self path = [self.path] try: file, filename, etc = imp.find_module(subname, path) except ImportError: return None return ImpLoader(fullname, file, filename, etc) Feh. The above won't properly handle the case where there *is* an __init__ module. Trying again: def find_module(self, fullname, path=None): subname = fullname.split(.)[-1] path = [self.path] try: file, filename, etc = imp.find_module(subname, path) except ImportError: if os.path.isdir(os.path.join(self.path, subname)): return self else: return None return ImpLoader(fullname, file, filename, etc) There, that should only fall back to __init__-less handling if there's no foo.py or foo/__init__.py present. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Guido van Rossum [EMAIL PROTECTED] wrote: On 4/26/06, Benji York [EMAIL PROTECTED] wrote: Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. I don't particularly like it. You still need __init__.py's for 'import *' to work (not that I like or use 'import *' :). The first question that pops into my mind when I think file-less modules is where does the package-module come from. That question is a lot easier to answer (not to mention explain) when all packages have an __init__.py. It also adds to Python's consistency (which means people learn something from it that they can apply to other things later; in that case, removing it just hampers their growth.) And besides, it's just not that big a deal. I don't feel strongly enough about it to object, though. However, I would suggest adding a warning for existing, __init__.py-less directories that would-have-been imported in 2.5. (There's an alpha3 scheduled, so it doesn't have to go in alpha2 tonight, and it could probably be last-minuted into beta1 too.) That should fix both Google's problems and that of everyone having existing non-package subdirs :-) Then, if it really matters, we can change the import in 2.6.Note that absolute import (which will be turned on for all in 2.6) 2.7, see the PEP. -- Thomas Wouters [EMAIL PROTECTED]Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Sounds a bit like the tail wagging the dog. I thought the Google geeks were a smart bunch. ISTM that something like Phillip Eby's code would be the most expedient solution. I would add one extension: if a package directory without an __init__.py file *is* encountered, an empty __init__.py file should automatically be created (and perhaps even svn add or equivalent called), and the code should loudly complain Packages need __init__.py files, noob! Add sound and light effects for extra credit. -- David Goodger http://python.net/~goodger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, André Malo [EMAIL PROTECTED] wrote: * Guido van Rossum wrote: On 4/26/06, André Malo [EMAIL PROTECTED] wrote: * Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. Not that it would count in any way, but I'd prefer to keep it. How would I mark a subdirectory as not-a-package otherwise? What's the use case for that? Have you run into this requirement? And even if you did, was there a requirement that the subdirectory's name be the same as a standard library module? If the subdirectory's name is not constrained, the easiest way to mark it as a non-package is to put a hyphen or dot in its name; if you can't do that, at least name it something that you don't need to import. Actually I have no problems with the change from inside python, but from the POV of tools, which walk through the directories, collecting/separating python packages and/or supplemental data directories. It's an explicit vs. implicit issue, where implicit would mean kind of heuristics from now on. IMHO it's going to break existing stuff [1] and should at least not be done in such a rush. nd [1] Well, it does break some of mine ;-) Can you elaborate? You could always keep the __init__.py files, you know... -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Phillip J. Eby [EMAIL PROTECTED] wrote: At 10:16 AM 4/26/2006 -0700, Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. Note that many tools exist which have grown to rely on the presence of __init__ modules. Also, although your proposal would allow imports to work reasonably well, tools that are actively looking for packages would need to have some way to distinguish package directories from others. My counter-proposal: to be considered a package, a directory must contain at least one module (which of course can be __init__). This allows the is it a package? question to be answered with only one directory read, as is the case now. Think of it also as a nudge in favor of flat is better than nested. I'm not sure what you mean by one directory read. You'd have to list the entire directory, which may require reading more than one block if the directory is large. But I'd be happy to define it like this from the POV of tools that want to know about sub-packages; my users complain because they have put .py files in a directory that they consider a sub-package so it would work fine for them. Python itself might attempt to consider the directory as a package and raise ImportError because the requested sub-module isn't found; the creation of a dummy entry in sys.modules in that case doesn't bother me. This tweak would also make it usable for top-level directories, since the mere presence of a 'time' directory wouldn't get in the way of anything. Actually, no; the case I remember was a directory full of Python code (all experiments by the user related to a particular topic -- I believe it was string). The thing more likely to have potential for problems is that many Python projects have a test directory that isn't intended to be a package, and thus may interfere with imports from the stdlib 'test' package. Whether this is really a problem or not, I don't know. test is a top-level package. I'm not proposing to change the rules for toplevel packages. Now you have the reason why. (And the new absolute import feature in 2.6 will prevent aliasing problems between subdirectories and top-level modules.) But, we could treat packages without __init__ as namespace packages. That is, set their __path__ to encompass similarly-named directories already on sys.path, so that the init-less package doesn't interfere with other packages that have the same name. Let's stick to the one feature I'm actually proposing please. This would require a bit of expansion to PEP 302, but probably not much. Most of the rest is existing technology, and we've already begun migrating stdlib modules away from doing their own hunting for __init__ and other files, towards using the pkgutil API. By the way, one small precedent for packages without __init__: setuptools generates such packages using .pth files when a package is split between different distributions but are being installed by a system packaging tool. In such cases, *both* parts of the package can't include an __init__, because the packaging tool (e.g. RPM) is going to complain that the shared file is a conflict. So setuptools generates a .pth file that creates a module object with the right name and initializes its __path__ to point to the __init__-less directory. This should be a small change. Famous last words. :) There's a bunch of tools that it's not going to work properly with, and not just in today's stdlib. (Think documentation tools, distutils extensions, IDEs...) Are you worried about the tools not finding directories that are now subpackages? Then fix the tools. Or are you worried about flagging subdirectories as (empty) packages since they exist, have a valid name (no hyphens, dots etc.) and contain no modules? I'm not sure I would call that failing. I can't see how a tool would crash or produce incorrect results with this change, *unless* you consider it incorrect to list a data directory as an empty package. To me, that's an advantage. Are you sure you wouldn't rather just write a GoogleImporter class to fix this problem? No, because that would require more setup code with a requirement to properly enable it, etc., etc., more failure modes, etc., etc. Append it to sys.path_hooks, clear sys.path_importer_cache, and you're all set. For that matter, if you have only one top-level package, put the class and the installation code in that top-level __init__, and you're set to go. I wish it were that easy. If there was such an easy solution, there wouldn't be pitchforks involved. I can't go into the details, but that just wouldn't work; and the problem happens most frequently to people who are already overloaded with learning new stuff. This is just one more bit of insanity they have to deal with. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum wrote: On 4/26/06, Barry Warsaw [EMAIL PROTECTED] wrote: On Wed, 2006-04-26 at 10:16 -0700, Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. I'm not really sure what this would buy us. Newbies would still forget the __init__.py in top-level packages (not all newbies work for Google). Oldies would have trouble recognizing a directory as being a Python package, rather than just a collection of modules - you wouldn't go hunting up the path to find the top-level __init__.py file which identifies the directory as being a sub-package of some top-level package (not all oldies work for Google either where you only have a single top-level package). -1. It doesn't appear to make things easier and breaks symmetry. But if it's there, then nothing changes, right? IOW, if we want to expose names in the subpackage's namespace, we can still do it in the subpackage's __init__.py. It's just that otherwise empty subpackage __init__.py files won't be required. Correct. This is an important clarification. What would the following print? import toplevel.sub1.sub2 print toplevel.sub1.sub2.__file__ If it's path/sub1/sub2 then that kind of breaks a common idiom of using os.path.dirname() on a module's __file__ to find co-located resources. Or at least, you have to know whether to dirname its __file__ or not (which might not be too bad, since you'd probably know how that package dir is organized anyway). Oh, cool gray area. I propose that if there's no __init__.py it prints 'path/sub1/sun2/' i.e. with a trailing slash; that causes dirname to just strip the '/'. (It would be a backslash on Windows of course). I dunno. Occasionally it trips me up too, but it's such an obvious and easy fix that it's never bothered me enough to care. But you're not a newbie. for a newbie who's just learned about packages, is familiar with Java, and missed one line in the docs, it's an easy mistake to make and a tough one to debug. Why not make the ImportError's text a little smarter instead, e.g. let the import mechanism check for this particular gotcha ? This would solve the newbie problem without any changes to the import scheme. I can't think of an example, but I suppose it's still possible that lifting this requirement could cause some in-package located data directories to be mistakenly importable. I'd be somewhat more worried about frameworks that dynamically import things having to be more cautious about cleansing their __import__() arguments now. But (assuming 2.6 and absolute import) what would be the danger of importing such a package? Presumably it contains no *.py or *.so files so there's no code there; but even so you'd have to go out of your way to import it (since if the directory exists it can't also be a subpackage or submodule name that's in actual use). Wasn't there agreement on postponing the absolute imports to Py3k due to the common use-case of turning e.g. 3rd party top-level packages into sub-packages of an application ? Without absolute imports your proposed scheme is going to break, since can easily mask top-level packages or modules. I'd be -1 but the remote possibility of you being burned at the stake by your fellow Googlers makes me -0 :). I'm not sure I understand what your worry is. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 26 2006) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
* Guido van Rossum wrote: [me] Actually I have no problems with the change from inside python, but from the POV of tools, which walk through the directories, collecting/separating python packages and/or supplemental data directories. It's an explicit vs. implicit issue, where implicit would mean kind of heuristics from now on. IMHO it's going to break existing stuff [1] and should at least not be done in such a rush. nd [1] Well, it does break some of mine ;-) [Guido] Can you elaborate? You could always keep the __init__.py files, you know... Okay, here's an example. It's about a non-existant __init__.py, though ;-) I have a test system which collects the test suites from one or more packages automatically by walking through the tree. Now there are subdirectories which explicitly are not packages (no __init__.py), but do contain some python files (helper scripts, spawned for particular tests). The test collector doesn't consider now these subdirectories at all, but in future it would need to (because it should search like python itself). Another point is that one can even hide supplementary packages within such a subdirectory. It's only visible to scripts inside the dir (I admit, that the latter is not a real usecase, just a thought that came up while writing this up). Anyway: sure, one could tweak the naming - just not for existing a.k.a. already released stuff. It's not very nice to force that, too ;-) nd -- If God intended people to be naked, they would be born that way. -- Oscar Wilde ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum wrote: On 4/26/06, Phillip J. Eby [EMAIL PROTECTED] wrote: At 10:16 AM 4/26/2006 -0700, Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. Note that many tools exist which have grown to rely on the presence of __init__ modules. Also, although your proposal would allow imports to work reasonably well, tools that are actively looking for packages would need to have some way to distinguish package directories from others. My counter-proposal: to be considered a package, a directory must contain at least one module (which of course can be __init__). This allows the is it a package? question to be answered with only one directory read, as is the case now. Think of it also as a nudge in favor of flat is better than nested. I'm not sure what you mean by one directory read. You'd have to list the entire directory, which may require reading more than one block if the directory is large. Sounds like a lot of filesystem activity to me :-) Currently, __init__.py is used as landmark by the import code to easily determine whether a directory is a package using two simple I/O operations (fstat on __init__.py, __init__.py[co]). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 26 2006) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On Wednesday 26 April 2006 15:05, André Malo wrote: Another point is that one can even hide supplementary packages within such a subdirectory. It's only visible to scripts inside the dir (I admit, that the latter is not a real usecase, just a thought that came up while writing this up). I have tests that do this. This is a very real use case. -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum wrote: On 4/26/06, Barry Warsaw [EMAIL PROTECTED] wrote: On Wed, 2006-04-26 at 10:16 -0700, Guido van Rossum wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. [...] I'd be -1 but the remote possibility of you being burned at the stake by your fellow Googlers makes me -0 :). I'm not sure I understand what your worry is. I happen to be a Googler too, but I was a Pythonista first... I'm -1 for minor mainly subjective reasons; 1) explicit is better than implicit. I prefer to be explicit about what is and isn't a module. I have plenty of doc and test and other directories inside python module source tree's that I don't want to be python modules. 2) It feels more consistant to always require it. /foo/ is a python package because it contains an __init__.py... so package /foo/bar/ should have one one too. 3) It changes things for what feels like very little gain. I've never had problems with it, and don't find the import exception hard to diagnose. Note that I think the vast majority of newbie missing __init__.py problems within google occur because people are missing __init__.py at the root of package import tree. This change would not not solve that problem. It wouldn't surprise me if this change would introduce a slew of newbies complaining that I have /foo on my PYTHONPATH, why can't I import foo/bar/ because they're forgotten the (now) rarely required __init__.py -- Donovan Baarda ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 11:50 AM 4/26/2006 -0700, Guido van Rossum wrote: I'm not sure what you mean by one directory read. You'd have to list the entire directory, which may require reading more than one block if the directory is large. You have to do this to find an __init__.py too, don't you? Technically, there's going to be a search for a .pyc or .pyo first, anyway. I'm just saying you can stop as soon as you hit an extension that's in imp.get_suffixes(). Are you sure you wouldn't rather just write a GoogleImporter class to fix this problem? No, because that would require more setup code with a requirement to properly enable it, etc., etc., more failure modes, etc., etc. I don't understand. I thought you said you had only *one* top-level package. Fix *that* package, by putting the code in its __init__.py. Job done. Append it to sys.path_hooks, clear sys.path_importer_cache, and you're all set. For that matter, if you have only one top-level package, put the class and the installation code in that top-level __init__, and you're set to go. I wish it were that easy. Well, if there's more than one top-level package, put the code in a module called google_imports and import google_import in each top-level package's __init__.py. I'm not sure I understand why a solution that works with released versions of Python that allows you to do exactly what you want, is inferior to a hypothetical solution for an unreleased version of Python that forces everybody else to update their tools. Unless of course the problem you're trying to solve is a political one rather than a technical one, that is. Or perhaps it wasn't clear from my explanation that my proposal will work the way you need it to, or I misunderstand what you're trying to do. Anyway, I'm not opposed to the idea of supporting this in future Pythons, but I definitely think it falls under the but sometimes never is better than RIGHT now rule where 2.5 is concerned. :) In particular, I'm worried that you're shrugging off the extent of the collateral damage here, and I'd be happiest if we waited until 3.0 before changing this particular rule -- and if we changed it in favor of namespace packages, which will more closely match naive user expectations. However, the fix the tools argument is weak, IMO. Zipfile imports have been a fairly half-assed feature for 2.3 and 2.4 because nobody took the time to make the *rest* of the stdlib work with zip imports. It's not a good idea to change machinery like this without considering at least what's going to have to be fixed in the stdlib. At a minimum, pydoc and distutils have embedded assumptions regarding __init__ modules, and I wouldn't be surprised if ihooks, imputil, and others do as well. If we can't keep the stdlib up to date with changes in the language, how can we expect anybody else to keep their code up to date? Finally, as others have pointed out, requiring __init__ at the top level just means that this isn't going to help anybody but Google. ISTM that in most cases outside Google, Python newbies are more likely to be creating top-level packages *first*, so the implicit __init__ doesn't help them. So, to summarize: 1. It only really helps Google 2. It inconveniences others who have to update their tools in order to support people who end up using it (even if by accident) 3. It's not a small change, unless you leave the rest of the stdlib unreviewed for impacts 4. It could be fixed at Google by adding a very small amount of code to the top of your __init__.py files (although apparently this is prevented for mysterious reasons that can't be shared) What's not to like? ;) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Phillip J. Eby wrote: My counter-proposal: to be considered a package, a directory must contain at least one module (which of course can be __init__). This allows the is it a package? question to be answered with only one directory read, as is the case now. Think of it also as a nudge in favor of flat is better than nested. I assume you want import x.y to fail if y is an empty directory (or non-empty, but without .py files). I don't see a value in implementing such a restriction. If there are no .py files in a tree, then there would be no point in importing it, so applications will typically not import an empty directory. Implementing an expensive test that will never give a positive result and causes no problems if skipped should be skipped. I can't see the problem this would cause to tools: they should assume any subdirectory can be a package, with all consequences this causes. If the consequences are undesirable, users should just stop putting non-package subdirectories into a package if they want to use the tool. However, I doubt there are undesirable consequences (although consequences might be surprising at first). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Guido van Rossum [EMAIL PROTECTED] wrote: So I have a very simple proposal: keep the __init__.py requirement for top-level pacakages, but drop it for subpackages. This should be a small change. I'm hesitant to propose *anything* new for Python 2.5, so I'm proposing it for 2.6; if Neal and Anthony think this would be okay to add to 2.5, they can do so. -1 from me. I had never a problem with __init__.py to mark a package or subpackage. Better add __namespace__.py to state a package dir as namespace package. And support multiple occurrences of on_python_path/namespace_name/package. -- bye by Wolfgang ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 09:56 PM 4/26/2006 +0200, Martin v. Löwis wrote: Phillip J. Eby wrote: My counter-proposal: to be considered a package, a directory must contain at least one module (which of course can be __init__). This allows the is it a package? question to be answered with only one directory read, as is the case now. Think of it also as a nudge in favor of flat is better than nested. I assume you want import x.y to fail if y is an empty directory (or non-empty, but without .py files). I don't see a value in implementing such a restriction. No, I'm saying that tools which are looking for packages and asking, Is this directory a package? should decide no in the case where it contains no modules. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Phillip J. Eby wrote: At 11:50 AM 4/26/2006 -0700, Guido van Rossum wrote: I'm not sure what you mean by one directory read. You'd have to list the entire directory, which may require reading more than one block if the directory is large. You have to do this to find an __init__.py too, don't you? Technically, there's going to be a search for a .pyc or .pyo first, anyway. No. Python does stat(2) and open(2) to determine whether a file is present in a directory. Whether or not that causes a full directory scan depends on the operating system. On most operating systems, it is *not* a full directory scan: - on network file systems, the directory is read only on the server; a full directory read would also cause a network transmission of the entire directory contents - on an advanced filesystem (such as NTFS), a lookup operation is a search in a balanced tree, rather than a linear search, bypassing many directory blocks for a large directory - on an advanced operating system (such as Linux), a repeated directory lookup for the file will not go to the file system each time, but cache the result of the lookup in an efficient memory structure. In all cases, the directory contents (whether read from disk into memory or not) does not have to be copied into python's address space for stat(2), but does for readdir(3). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Phillip J. Eby wrote: I assume you want import x.y to fail if y is an empty directory (or non-empty, but without .py files). I don't see a value in implementing such a restriction. No, I'm saying that tools which are looking for packages and asking, Is this directory a package? should decide no in the case where it contains no modules. Ah. Tools are of course free to do that. It would slightly deviate from Python's implementation of import, but the difference wouldn't matter for all practical purposes. So from a language lawyers' point of view, I would specify: A sub-package is a sub-directory of a package that contains at least one module file. Python implementations MAY accept sub-directories as sub-packages even if they contain no module files as package, instead of raising ImportError on attempts to import that sub-package. (a module file, in that context, would be a file file which matches imp.get_suffixes(), in case that isn't clear) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] (Context: There's a large crowd with pitchforks and other sharp pointy farm implements just outside the door of my office at Google. They are making an unbelievable racket. It appears they are Google engineers who have been bitten by a misfeature of Python, and they won't let me go home before I have posted this message.) One particular egregious problem is that *subpackage* are subject to the same rule. It so happens that there is essentially only one top-level package in the Google code base, and it already has an __init__.py file. But developers create new subpackages at a frightening rate, and forgetting to do touch __init__.py has caused many hours of lost work, not to mention injuries due to heads banging against walls. It seems to me that the right way to fix this is to simply make a small change to the error message. On a failed import, have the code check if there is a directory that would have been the requested package if it had contained an __init__ module. If there is then append a message like You might be missing an __init__.py file. It might also be good to check that the directory actually contained python modules. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
OK, forget it. I'll face the pitchforks. I'm disappointed though -- it sounds like we can never change anything about Python any more because it will upset the oldtimers. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 04:33 PM 4/26/2006 -0400, Joe Smith wrote: It seems to me that the right way to fix this is to simply make a small change to the error message. On a failed import, have the code check if there is a directory that would have been the requested package if it had contained an __init__ module. If there is then append a message like You might be missing an __init__.py file. It might also be good to check that the directory actually contained python modules. This is a great idea, but might be hard to implement in practice with the current C implementation of import, at least for the general case. But if we're talking about subpackages only, the common case is a one-element __path__, and for that case there might be something we could do. (The number of path items is relevant because the existence of a correctly-named but init-less directory should not stop later path items from being searched, so the actual error occurs far from the point where the empty directory would've been detected.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 01:49 PM 4/26/2006 -0700, Guido van Rossum wrote: OK, forget it. I'll face the pitchforks. I'm disappointed though -- it sounds like we can never change anything about Python any more because it will upset the oldtimers. I know exactly how you feel. :) But there's always Python 3.0, and if we're refactoring the import machinery there, we can do this the right way, not just the right now way. ;) IMO, if Py3K does this, it can and should be inclusive of top-level packages and assemble __path__ using all the sys.path entries. If we're going to break it, let's break it all the way. :) I'm still really curious why the importer solution (especially if tucked away in a Google-defined sitecustomize) won't work, though. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Terry Jones [EMAIL PROTECTED] wrote: I would suggest adding a hook to their version control system to automatically create (and preferably also check out) an __init__.py file whenever a new (source code) directory was placed under version control (supposing you can distinguish source code directories from the check in dirname). This wouldn't work of course -- the newbie would try to test it before checking it in, so the hook would not be run. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Phillip J. Eby [EMAIL PROTECTED] wrote: At 01:49 PM 4/26/2006 -0700, Guido van Rossum wrote: OK, forget it. I'll face the pitchforks. I'm disappointed though -- it sounds like we can never change anything about Python any more because it will upset the oldtimers. I know exactly how you feel. :) Hardly -- you're not the BDFL. :) But there's always Python 3.0, and if we're refactoring the import machinery there, we can do this the right way, not just the right now way. ;) IMO, if Py3K does this, it can and should be inclusive of top-level packages and assemble __path__ using all the sys.path entries. If we're going to break it, let's break it all the way. :) No -- I'm actually quite happy with most of the existing behavior (though not with the APIs). I'm still really curious why the importer solution (especially if tucked away in a Google-defined sitecustomize) won't work, though. Well, we have a sitecustomize, and it's been decided that it's such a pain to get it to do the right thing that we're trying to get rid of it. So proposing to add more magic there would not work. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On Apr 26, 2006, at 4:49 PM, Guido van Rossum wrote: OK, forget it. I'll face the pitchforks. I'm disappointed though -- it sounds like we can never change anything about Python any more because it will upset the oldtimers. No, you can not make a change which has a tiny (and arguably negative) advantage but large compatibility risks. Like it or not, Python is a stable, widely deployed program. Making almost arbitrary sideways changes is not something you can do to such a system without a lot of pushback. Breaking things requires (and *should* require) well thought out reasoning as to why the new way is actually better enough to justify the breakage. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Phillip J. Eby [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] It might also be good to check that the directory actually contained python modules. This is a great idea, but might be hard to implement in practice with the current C implementation of import, at least for the general case. Perhaps checking and autogeneration of __init__.py should better be part of a Python-aware editor/IDE. A File menu could have a New Package entry to create a directory + empty __init__.py. Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Joe Smith wrote: It seems to me that the right way to fix this is to simply make a small change to the error message. On a failed import, have the code check if there is a directory that would have been the requested package if it had contained an __init__ module. If there is then append a message like You might be missing an __init__.py file. +1. It's not that putting an __init__.py file in is hard, it's that people have a hard time realizing when they've forgotten to do it. -- Ian Bicking / [EMAIL PROTECTED] / http://blog.ianbicking.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Guido van Rossum [EMAIL PROTECTED] wrote: OK, forget it. I'll face the pitchforks.Maybe this'll help:http://python.org/sf/1477281 (You can call it 'oldtimer-repellant' if you want to use it to convince people there isn't any *real* backward-compatibility issue.) I'm disappointed though -- it sounds like we can never change anythingabout Python any more because it will upset the oldtimers. That's a bit unfair, Guido. There are valid reasons not to change Python's behaviour in this respect, regardless of upset old-timers. Besides, you're the BDFL; if you think the old-timers are wrong, I implore you to put their worries aside (after dutiful contemplation.) I've long since decided that any change what so ever will have activist luddites opposing it. I think most of them would stop when you make a clear decision -- how much whining have you had about the if-else syntax since you made the choice? I've heard lots of people gripe about it in private (at PyCon, of course, I never see Pythonistas anywhere else :-P), but I haven't seen any python-dev rants about it. I certainly hate PEP-308's guts, but if if-else is your decision, if-else is what we'll do. And so it is, I believe, with this case. -- Thomas Wouters [EMAIL PROTECTED]Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Thomas Wouters [EMAIL PROTECTED] wrote: On 4/26/06, Guido van Rossum [EMAIL PROTECTED] wrote: OK, forget it. I'll face the pitchforks. Maybe this'll help: http://python.org/sf/1477281 (You can call it 'oldtimer-repellant' if you want to use it to convince people there isn't any *real* backward-compatibility issue.) I'd worry that it'll cause complaints when the warning is incorrect and a certain directory is being skipped intentionally. E.g. the string directory that someone had. Getting a warning like this can be just as upsetting to newbies! I'm disappointed though -- it sounds like we can never change anything about Python any more because it will upset the oldtimers. That's a bit unfair, Guido. There are valid reasons not to change Python's behaviour in this respect, regardless of upset old-timers. Where are the valid reasons? All I see is knee-jerk -1, -1, -1, and this might cause tools to do the wrong thing. Not a single person attempted to see it from the newbie POV; several people explicitly rejected the newbie POV as invalid. I still don't know the name of any tool that would break due to this *and where the breakage wouldn't be easy to fix by adjusting the tool's behavior*. Yes, fixing tools is a pain. But they have to be fixed for every new Python version anyway -- new keywords, new syntax, new bytecodes, etc. Besides, you're the BDFL; if you think the old-timers are wrong, I implore you to put their worries aside (after dutiful contemplation.) I can only do that so many times before I'm no longer the BDFL. It's one thing to break a tie when there is widespread disagreement amongst developers (like about the perfect decorator syntax). It's another to go against a see of -1's. I've long since decided that any change what so ever will have activist luddites opposing it. I think most of them would stop when you make a clear decision -- how much whining have you had about the if-else syntax since you made the choice? I've heard lots of people gripe about it in private (at PyCon, of course, I never see Pythonistas anywhere else :-P), but I haven't seen any python-dev rants about it. I certainly hate PEP-308's guts, but if if-else is your decision, if-else is what we'll do. And so it is, I believe, with this case. OK. Then I implore you, please check in that patch (after adding error checking for PyErr_Warn() -- and of course after a2 hs been shipped), and damn the torpedoes. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/27/06, Guido van Rossum [EMAIL PROTECTED] wrote: On 4/26/06, Thomas Wouters [EMAIL PROTECTED] wrote: On 4/26/06, Guido van Rossum [EMAIL PROTECTED] wrote: OK, forget it. I'll face the pitchforks. Maybe this'll help: http://python.org/sf/1477281 (You can call it 'oldtimer-repellant' if you want to use it to convince people there isn't any *real* backward-compatibility issue.)I'd worry that it'll cause complaints when the warning is incorrectand a certain directory is being skipped intentionally. E.g. thestring directory that someone had. Getting a warning like this can be just as upsetting to newbies!I don't think getting a spurious warning is as upsetting as getting no warning but the damned thing just not working. At least you have something to google for. And the warning includes the original line of source that triggered it *and* the directory (or directories) it's complaining about, which is quite a lot of helpful hints. The clashes with directories that aren't intended to be packages *and* a module of the same name is imported, yes, that's a real problem. It's not any worse than if we change package imports the way you originally proposed, though, and I think the actual number of spurious errors is very small (which self-respecting module still does 'import string', eh? :-) I don't think the fix for such a warning is going to be non-trivial. Where are the valid reasons?Of course, I only consider *my* reasons to be valid, and mine weren't knee-jerk or tool-related. I don't think Python should be going Oh, what you wanted wasn't possible, but I think I know what you wanted, let me do it for you, first of all because it's not very Pythonic, and second of all because it doesn't lower the learning curve, it just delays some upward motion a little (meaning the curve may become steeper, later.) A clear warning, on the other hand, can be a helpful nudge towards the 'a-HA' moment. Then I implore you, please check in that patch (after adding errorchecking for PyErr_Warn() -- and of course after a2 hs been shipped), and damn the torpedoes.Alrighty then. The list has about 12 hours to convince me (and you) that it's a bad idea to generate that warning. I'll be asleep by the time the trunk un-freezes, and I have a string of early meetings tomorrow. I'll get to it somewhere in the afternoon :) -- Thomas Wouters [EMAIL PROTECTED]Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum wrote: http://python.org/sf/1477281 (You can call it 'oldtimer-repellant' if you want to use it to convince people there isn't any *real* backward-compatibility issue.) I'd worry that it'll cause complaints when the warning is incorrect and a certain directory is being skipped intentionally. E.g. the string directory that someone had. Getting a warning like this can be just as upsetting to newbies! I really think it would be more useful having an ImportError containing a suggestion than having a warning. Anyone who knows it's bogus can just ignore it. I'd probably make it that all directories that could have been imports get listed. FWIW I was definitely a kneejerk -1. After reading all the messages in this thread, I think I'm now a non-kneejerk -1. It seems like gratuitous change introducing inconsistency for minimal benefit - esp. if there is a notification that a directory *could* have been a package on import failure. I think it's a useful feature of Python that it's simple to distinguish a package from a non-package. Tim Delaney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Thomas Wouters [EMAIL PROTECTED] wrote: Of course, I only consider *my* reasons to be valid, and mine weren't knee-jerk or tool-related. I don't think Python should be going Oh, what you wanted wasn't possible, but I think I know what you wanted, let me do it for you, first of all because it's not very Pythonic, and second of all because it doesn't lower the learning curve, it just delays some upward motion a little (meaning the curve may become steeper, later.) A clear warning, on the other hand, can be a helpful nudge towards the 'a-HA' moment. That still sounds like old-timer reasoning. Long ago we were very close to defining a package as a directory -- with none of this must contain __init__.py or another *.py file nonsense. IIRC the decision to make __init__.py mandatory faced opposition too, since people were already doing packages with just directories (which is quite clean and elegant, and that's also how it was in Java), but I added it after seeing a few newbies tear out their hair. I believe that if at that time __init__.py had remained optional, and today I had proposed to require it, the change would have been derided as unpythonic as well. There's nothing particularly unpythonic about optional behavior; e.g. classes may or may not provide an __init__ method. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Delaney, Timothy (Tim) [EMAIL PROTECTED] wrote: I really think it would be more useful having an ImportError containing a suggestion than having a warning. Anyone who knows it's bogus can just ignore it. That's effectively what Thomas's patch does though -- if at the end the path the package isn't found, you'll still get an ImportError; but it will be preceded by ImportWarnings showing you the non-package directories found on the way. The difference is that if you find a valid module package later on the path, you'll still get warnings. But occasionally those will be useful too -- when you are trying to create a package that happens to have the same name as a standard library package or module, it's a lot easier to debug the missing __init__.py if you get a warning about it instead of having to figure out that the foo you imported is not the foo you intended. The symptom is usually a mysterious AttributeError that takes a bit of determination to debug. Of course print foo.__file__ usually sets you right, but that's not the first thing a newbie would try -- they aren't quite sure about all the rules of import, so they are likely to first try to fiddle with $PYTHONPATH or sys.path, then put print statements in their package, etc. I'd probably make it that all directories that could have been imports get listed. Thomas' patch does this automatically -- you get a warning for each directory that is weighed and found too light. FWIW I was definitely a kneejerk -1. After reading all the messages in this thread, I think I'm now a non-kneejerk -1. It seems like gratuitous change introducing inconsistency for minimal benefit - esp. if there is a notification that a directory *could* have been a package on import failure. I think it's a useful feature of Python that it's simple to distinguish a package from a non-package. The change is only gratuitous if you disagree with it. The inconsistency is real but we all know the line about a foolish consistency. The benefit is minimal unless you've wasted an hour debugging this situation. Is it also useful to distinguish a subpackage from a non-subpackage? Anyway, the warning is more compatible and just as helpful so we'll go with that. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Guido van Rossum wrote: The difference is that if you find a valid module package later on the path, you'll still get warnings. This is the bit I don't like about it. Currently the warnings are displayed as the packages are found. I'd be quite happy with the warnings if they were suppressed until there is an actual ImportError - at which time they'll be displayed. But occasionally those will be useful too -- when you are trying to create a package that happens to have the same name as a standard library package or module, it's a lot easier to debug the missing __init__.py if you get a warning about it instead of having to figure out that the foo you imported is not the foo you intended. I suspect that more often any warnings when there is a successful import will be spurious. But testing in the field will reveal that. So I say alpha 3 should have Thomas' patch as is, and it can be changed afterwards if necessary. Thomas' patch does this automatically -- you get a warning for each directory that is weighed and found too light. Not exactly - you'll get warnings for directories earlier in sys.path than a matching package. Potential packages later in the path won't be warned about. If you're trying to resolve import problems, it's just as likely that the package you really want is later in sys.path than earlier. Obviously in the case that you get an ImportError this goes away. Is it also useful to distinguish a subpackage from a non-subpackage? Possibly. Perhaps it would be useful to have `is_package(dirname)`, `is_rootpackage(dirname)` and `is_subpackage(dirname)` functions somewhere (pkgutils?). Then the tools objections go away, and whatever mechanism is necessary to determine this (e.g. is_rootpackage checks if the directory is importable via sys.path) can be implemented. Anyway, the warning is more compatible and just as helpful so we'll go with that. Fair enough. Tim Delaney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 01:10 AM 4/27/2006 +0200, Thomas Wouters wrote: On 4/27/06, Guido van Rossum mailto:[EMAIL PROTECTED][EMAIL PROTECTED] wrote: I'd worry that it'll cause complaints when the warning is incorrect and a certain directory is being skipped intentionally. E.g. the string directory that someone had. Getting a warning like this can be just as upsetting to newbies! I don't think getting a spurious warning is as upsetting as getting no warning but the damned thing just not working. At least you have something to google for. And the warning includes the original line of source that triggered it *and* the directory (or directories) it's complaining about, which is quite a lot of helpful hints. +1. If the warning is off-base, you can rename the directory or suppress the warning. As for the newbie situation, ISTM that this warning will generally come close enough in time to an environment change (new path entry, newly created conflicting directory) to be seen as informative. The only time it might be confusing is if you had just added an import foo after having a foo directory sitting around for a while. But even then, the warning is saying, hey, it looked like you might have meant *this* foo directory... if so, you're missing an __init__. So, at that point I rename the directory... or maybe add the __init__ and break my code. So then I back it out and put up with the warning and complain to c.l.p, or maybe threaten Guido with a pitchfork if I work at Google. Or maybe just a regular-sized fork, since the warning is just annoying. :) Alrighty then. The list has about 12 hours to convince me (and you) that it's a bad idea to generate that warning. I'll be asleep by the time the trunk un-freezes, and I have a string of early meetings tomorrow. I'll get to it somewhere in the afternoon :) I like the patch in general, but may I suggest PackageWarning or maybe BrokenPackageWarning instead of ImportWarning as the class name? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Delaney, Timothy (Tim) [EMAIL PROTECTED] wrote: Potential packages later in the path won't be warned about. If you're trying to resolve import problems, it's just as likely that the package you really want is later in sys.path than earlier. But module hiding is a feature, and anyway, we're not going to continue to search sys.path after we've found a valid match (imagine doing this on *every* successful import!), so you're not going to get warnings about those either way. Obviously in the case that you get an ImportError this goes away. Right. Is it also useful to distinguish a subpackage from a non-subpackage? Possibly. Perhaps it would be useful to have `is_package(dirname)`, `is_rootpackage(dirname)` and `is_subpackage(dirname)` functions somewhere (pkgutils?). YAGNI. Also note that not all modules or packages are represented by pathnames -- they could live in zip files, or be accessed via whatever other magic an import handler users. (Thomas's warning won't happen in those cases BTW -- it only affects the default import handler.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
At 04:57 PM 4/26/2006 -0700, Guido van Rossum wrote: On 4/26/06, Delaney, Timothy (Tim) [EMAIL PROTECTED] wrote: Possibly. Perhaps it would be useful to have `is_package(dirname)`, `is_rootpackage(dirname)` and `is_subpackage(dirname)` functions somewhere (pkgutils?). YAGNI. Also note that not all modules or packages are represented by pathnames -- they could live in zip files, or be accessed via whatever other magic an import handler users. FYI, pkgutil in 2.5 has utilities to walk a package tree, starting from sys.path or a package __path__, and it's PEP 302 compliant. pydoc now uses this in place of directory inspection, so that documenting zipped packages works correctly. These functions aren't documented yet, though, and probably won't be until next week at the earliest. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On Thursday 27 April 2006 05:50, Phillip J. Eby wrote: Anyway, I'm not opposed to the idea of supporting this in future Pythons, but I definitely think it falls under the but sometimes never is better than RIGHT now rule where 2.5 is concerned. :) I agree fully. I don't think we should try and shove this into Python 2.5 on short notice, but I could be convinced otherwise. Right now, though, I'm a strong -1 for now for this in 2.5. If it's to go forward, I think it _definitely_ needs a PEP outlining the potential breakages (and I'm not sure we're aware of them all yet). In particular, I'm worried that you're shrugging off the extent of the collateral damage here, and I'd be happiest if we waited until 3.0 before changing this particular rule -- and if we changed it in favor of namespace packages, which will more closely match naive user expectations. The breakage of tools and the like is my concern, too. Python's import machinery is already a delicate mess of subtle rules. -- Anthony Baxter [EMAIL PROTECTED] It's never too late to have a happy childhood. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On Thursday 27 April 2006 06:49, Guido van Rossum wrote: OK, forget it. I'll face the pitchforks. I'm disappointed though -- it sounds like we can never change anything about Python any more because it will upset the oldtimers. I'm not averse to changing this - just not to changing it on short notice for 2.5 and causing me pain from having to cut new releases to fix some breakage in the stdlib caused by this wink -- Anthony Baxter [EMAIL PROTECTED] It's never too late to have a happy childhood. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Anthony Baxter [EMAIL PROTECTED] wrote: On Thursday 27 April 2006 06:49, Guido van Rossum wrote: OK, forget it. I'll face the pitchforks. I'm disappointed though -- it sounds like we can never change anything about Python any more because it will upset the oldtimers. I'm not averse to changing this - just not to changing it on short notice for 2.5 and causing me pain from having to cut new releases to fix some breakage in the stdlib caused by this wink I wasn't proposing it for 2.5 (for this very reason) -- AFAICT most responses were again st this for 2.6 or 2.7. Hopefully you're okay with Thomas's patch going into 2.5a3 -- it *warns* about directories without __init__.py that otherwise match the requested name. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Boy, threads here sure move fast when there's work to be done :). Although largely moot now, I'll follow up for posterity's sake. On Wed, 2006-04-26 at 10:59 -0700, Guido van Rossum wrote: Oh, cool gray area. I propose that if there's no __init__.py it prints 'path/sub1/sun2/' i.e. with a trailing slash; that causes dirname to just strip the '/'. (It would be a backslash on Windows of course). Yep, that's the right thing to do. Given all the other traffic in this thread, I don't have much more to add except: I probably don't remember things accurately, but ISTR that __init__.py was added as a way to expose names in the package's namespace first. We probably went from that to requiring __init__.py as a way to be explicit about what directories are packages and which aren't. So I suspect you're right when you say that if the rule had already been relaxed and you were now proposing to tighten the rules, we probably get just as many complaints. alternative-universe-ly y'rs, -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On Wednesday 26 April 2006 22:42, Barry Warsaw wrote: So I suspect you're right when you say that if the rule had already been relaxed and you were now proposing to tighten the rules, we probably get just as many complaints. Indeed. I think the problem many of us have with the proposal isn't the new behavior, but the change in the behavior. That's certainly it for me. -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On 4/26/06, Fred L. Drake, Jr. [EMAIL PROTECTED] wrote: Indeed. I think the problem many of us have with the proposal isn't the new behavior, but the change in the behavior. That's certainly it for me. Which is why I said earlier that I felt disappointed that we can't change anything any more. But I'm fine with the warning -- it should be enough to keep Google's newbies from wasting their time. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
On Wed, Apr 26, 2006, Guido van Rossum wrote: Which is why I said earlier that I felt disappointed that we can't change anything any more. I've been here since Python 1.5.1. I don't understand why this issue in particular makes you feel disappointed. I also think your statement is just plain untrue. Looking at the What's New for Python 2.5, we have changes to the import machinery, the with statement, new functionality for generators, conditional expressions, ssize_t, exceptions as new-style classes, deleted modules (regex, regsub, and whrandom), and on and on and on. Some of these changes are going to cause moderate breakage for some people; they cumulatively represent a *lot* of change. Python 2.5 is gonna be a *huge* release. Quite honestly, I think you've let yourself get emotionally attached to this issue for some reason. On the other hand, I think that a lot of the blowback you got comes from you bringing up this issue right before 2.5a2. My suggestion is that you let this sit and make yourself a calendar entry to think about this again in October. If it still seems like a good idea, go ahead and bring it up. (Or just punt to 3.0.) -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ Argue for your limitations, and sure enough they're yours. --Richard Bach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
