Re: [Python-Dev] Thoughts on stdlib evolvement
Fernando Perez wrote: > Josiah Carlson wrote: >>Fernando Perez wrote: >>>I've wondered if it wouldn't be better if the std lib were all stuffed into >>>its own namespace: >>> >>>from std import urllib >>> >>>If a more structured approach is desired, it could be >>> >>>from std.www import urllib >> >>This generally brings up the intersection of stdlib and nonstdlib naming >>hierarchy. More specifically, what does "import email" mean? >>Presumably it means to import the email module or package, but from the >>current module directory, or from the standard library? > > > Well, I've thought of this (ligthly) mostly as a py3k thing, since it would > require that 'import email' in the naked fails, as it would become 'import > std.email', or 'import std.www.email' or whatever. A plain 'import email' > would then refer to some third-party 'email' module, not part of the standard > library. > > Since this would mean a massive break of exisiting code, it would necessarily > be > a py3k issue. But nonetheless the idea of confinign the stdlib to the 'std' > namespace does have some appeal, at least to me. Actually no. Re: std.* If there were a single path (or set of paths) know at install time where standard library modules are, then that can be placed in a PYSDTLIB environment variable (or some other specified place). Inside Python, whenever an import is done relative to std, one merely examines the paths offered via the PYSTDLIB environment variable for the looked-for module. For modules/packages in site-packages, one could use another name, perhaps "packages" or somesuch, to refer to it. sys.path is not usable as-is because it can contain the path of the module named __main__, and is (not uncommonly) modified by user code. This can be used with a proper import hook, which is turned on per-module via a __future__ import, hence no backwards compatibility issues. What I had been working on is the use of special import semantics for non-stdlib/site-packages modules. Specifically if you were to write 'mymodule.py' and import your other module 'myothermodule.py' in the same path, you would use "from . import myothermodule". It doesn't solve any problems with standard library naming, but it does allow one to do cousin imports... "from ..uncle import cousin", as well as all sorts of other things, which allows (in writing software) for modules and packages to be shared by other packages. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts on stdlib evolvement
Josiah Carlson wrote: > Fernando Perez wrote: >> I've wondered if it wouldn't be better if the std lib were all stuffed into >> its own namespace: >> >> from std import urllib >> >> If a more structured approach is desired, it could be >> >> from std.www import urllib > > This generally brings up the intersection of stdlib and nonstdlib naming > hierarchy. More specifically, what does "import email" mean? > Presumably it means to import the email module or package, but from the > current module directory, or from the standard library? Well, I've thought of this (ligthly) mostly as a py3k thing, since it would require that 'import email' in the naked fails, as it would become 'import std.email', or 'import std.www.email' or whatever. A plain 'import email' would then refer to some third-party 'email' module, not part of the standard library. Since this would mean a massive break of exisiting code, it would necessarily be a py3k issue. But nonetheless the idea of confinign the stdlib to the 'std' namespace does have some appeal, at least to me. Best, f ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts on stdlib evolvement
> "Skip" == Skip Montanaro <[EMAIL PROTECTED]> writes: Skip> If you provide the necessary namespace structure for them to Skip> nestle into, I suspect most of them could be maintained Skip> outside the stdlib just fine. FWIW, this has worked well for XEmacs; it's one of our most popular features vis-a-vis GNU Emacs. Users like it because we do provide a centralized directory of modules (with many mirrors) that they can download updates from, as well as a "batteries included" library distribution (which we call "SUMO tarballs" for historical reasons). Package developers like it because they can maintain closer control, yet have up-to-date versions distributed from our repository. Finally the core maintainers like it because there's a clear delegation of responsibility. We provide a centralized repository, which is mirrored by a couple score sites; I'm not sure how much harder it would be to make it distributed. The essential feature is the centralized directory. We do have problems. One is the megapackage issue that Phillip Eby referred in his reply to Bob Ippolito <[EMAIL PROTECTED]>. Our packaging system doesn't work well for them. Some of the problems are our fault, but I think it's actually a hard problem; we could fix up some details, but not really address the core issues of megapackages. The other one is that our initial division into "core" and "package" was pretty radical, and several core developers still want to pull certain packages back into core. But this creates backward compatibility problems. I think this would be less of a problem for Python. Creating the package repository could be more conservative: Python's standard library has plenty of modules that would not present such problems. In any case, Python has better facilities for dealing with backward compatibility issues (such as deprecation warnings and the like) than we do, and the language itself is simply more suited to preserving modularity than Emacs Lisp is (with its single dynamically scoped namespace). Our package namespace structure is trivial (it's basically flat); what is more important than the namespace structure, I think, is provision of package organization and directory utilities as Phillip Eby mentions. If they're reasonably flexible and Python invests some effort in helping package maintainers with different current distribution methods to adapt, I think this would work well for Python too. One thing we have found is that it takes only a small amount of incremental work to maintain the package directory and repository as such, even as it has grown dramatically. By contrast, I find the Zope wiki-based organization of Products quite annoying to work with. Of course the packaged code itself can and does bitrot if the external maintainer goes AWOL, or simply doesn't like hanging out with XEmacs people. In such cases we've found that users will often step up to help with the nitty-gritty management of "their" package, such as tracking bug reports, liaison with an "unsociable" upstream, regression testing, and integrating changes (anybody can run make and patch), even if they have to ask for help with code-related work. Ie, the fact that there is a well-defined package requiring specific management tasks encourages them to take on the work they can handle. For XEmacs the effort of separating out the packages was moderately large; it took about a year, with three or four developers spending a substantial fraction of their time on it, either organizing the repository or creating infrastructure. It was a very worthwhile investment. Python's investment probably would be proportionately smaller, as much of the necessary infrastructure seems to be already available. For XEmacs, I think the main returns come from creating well-defined chunks that people can "own" and take responsibility for. This helps to mitigate the "dormant bug" problem, and generally keeps the packaged code fresher than it was in the old days, even though we have a much larger volume of code and probably less manpower now. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts on stdlib evolvement
Fernando Perez wrote: > Skip Montanaro wrote: > > >>I wouldn't mind a stdlib that defined a set of top-level packages (some of >>which might be wholly unpopulated by modules in the standard distribution) >>It might, for example, define a gui package and gui.Tkinter and gui._tkinter >>modules, leaving the remainder of gui namespace available for 3rd party >>modules. Such a scheme would probably work best if there was some fairly >>lightweight registration/approval process in the community to avoid needless >>collisions. For example, it might be a bit confusing if two organizations >>began installing their packages in a subpackage named gui.widgets. An >>unsuspecting person downloading an application that used one version of >>gui.widgets into environment with the conflicting gui.widgets would run into >>trouble. > > > I've wondered if it wouldn't be better if the std lib were all stuffed into > its > own namespace: > > from std import urllib > > If a more structured approach is desired, it could be > > from std.www import urllib This generally brings up the intersection of stdlib and nonstdlib naming hierarchy. More specifically, what does "import email" mean? Presumably it means to import the email module or package, but from the current module directory, or from the standard library? I posted about this months ago, and got to writing an import hook that used a method suggested by Guido by naming things (sys.modules names) based on whether they were imported relative to the module with a __name__ of '__main__', and using the prefix dot semantics for package-relative imports to mean any relative imports. I eventually got hung up on some nested import bits, and got busy with work. I may have time to hack on it next week to finish the 'proof of concept' (currently in the Bay area for work), but if someone wants or needs more explanation, I would be happy to offer it. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts on stdlib evolvement
Skip Montanaro wrote: > I wouldn't mind a stdlib that defined a set of top-level packages (some of > which might be wholly unpopulated by modules in the standard distribution) > It might, for example, define a gui package and gui.Tkinter and gui._tkinter > modules, leaving the remainder of gui namespace available for 3rd party > modules. Such a scheme would probably work best if there was some fairly > lightweight registration/approval process in the community to avoid needless > collisions. For example, it might be a bit confusing if two organizations > began installing their packages in a subpackage named gui.widgets. An > unsuspecting person downloading an application that used one version of > gui.widgets into environment with the conflicting gui.widgets would run into > trouble. I've wondered if it wouldn't be better if the std lib were all stuffed into its own namespace: from std import urllib If a more structured approach is desired, it could be from std.www import urllib for example. But having std. as the top-level namespace for the stdlib, could simplify (I think) life a lot in the long run. If a decision for a more structured namespace is made, then it might be nice to have the same top-level structure in site-packages, albeit empty by default: from std.www import foo -> standard library www packages from www import bar -> third-party www packages Third-party packages can still be put into base site-packages, of course, but developers could be encouraged to transition into putting things into these categories. This would also ease the process of 'staging' a module as external for a while before deciding whether it meets the requirement for being put into the stdlib. Just an idea (sorry if it's been discussed and shot down before). best, f ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts on stdlib evolvement
Tim> On 6/6/05, Reinhold Birkenfeld <[EMAIL PROTECTED]> wrote: >> - Flat namespace: Should we tend to a more hierarchic library (e.g. >> inet.url, inet.http, inet.nntp)? This would increase clarity when >> searching for a module. Tim> -1. I feel the opposite way: when trying to figure out where Tim> something "lives", I prefer Python's flat namespace to (for Tim> example) Java's com.company.baz.bar.foo hierarchy. I think Reinhold's suggestion (e.g., inet.*) was for a flatter namespace than Java's scheme, but for a slightly more structured namespace than the current standard library provides. Some amount of structure helps to collect modules together whose names don't obviously suggest who their neighbors are in the functionality space. For example, to the naive user it might not be obvious that these groups of modules and packages are related: * getopt and optparse * bsddb, gdbm and anydbm * email, mhlib, quopri * heapq, collections * user, site, sitecustomize * profile, hotshot, pstats * pipes, subprocess, os I wouldn't mind a stdlib that defined a set of top-level packages (some of which might be wholly unpopulated by modules in the standard distribution) It might, for example, define a gui package and gui.Tkinter and gui._tkinter modules, leaving the remainder of gui namespace available for 3rd party modules. Such a scheme would probably work best if there was some fairly lightweight registration/approval process in the community to avoid needless collisions. For example, it might be a bit confusing if two organizations began installing their packages in a subpackage named gui.widgets. An unsuspecting person downloading an application that used one version of gui.widgets into environment with the conflicting gui.widgets would run into trouble. Is the CPAN namespace wide open? If I wanted to create a Perl module to fit into the HTML namespace is there some procedure involved or is it an example of squatters' rights? >> - 3rd party modules: There are many superb modules out there, some of >> which really do have a "standard" character. Examples include PIL, >> numarray, ElementTree, [wxPython - I know this is a hairy issue], Tim> I think the most important question for each of these is "is the Tim> module's release schedule at least as stable as Python's?". For Tim> many of these, I suspect the answer is "no". If you provide the necessary namespace structure for them to nestle into, I suspect most of them could be maintained outside the stdlib just fine. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts on stdlib evolvement
At 03:17 PM 6/6/2005 -0700, Bob Ippolito wrote: >Personally I'd like to see the standard library get smaller rather >than larger. There's a whole lot of bit rot in there, and since >sys.path prefers the standard library over anything else it's a >really huge pain to integrate patches on a faster release schedule >than Python's while remaining sane at the same time. You know, before you said that, it hadn't occurred to me that the Python standard library is subject to the same economic forces that cause mega-packages like Twisted, SciPy, Zope, etc. to develop. Specifically, the cost incurred by relying on an externally-distributed dependency causes anyone with non-trivial needs to create "batteries included" libraries. One of my goals for Python Eggs and EasyInstall was to lower this dependency-cost barrier by reducing the "dependency cost" to zero at the point of installation, by making it as easy to install ten packages as one. (Another was to reduce the dependency cost for the developer, who need only add package metadata for the dependency to be fulfilled at installation time.) Now that you've pointed out the parallel between the stdlib and the other modules, I wonder if Python 3.0 might be able to take a more minimalist approach to the standard library, if it included the equivalents of easy_install and pkg_resources. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts on stdlib evolvement
On 6/6/05, Reinhold Birkenfeld <[EMAIL PROTECTED]> wrote: > - Flat namespace: Should we tend to a more hierarchic library (e.g. > inet.url, inet.http, inet.nntp)? This would increase clarity when > searching for a module. -1. I feel the opposite way: when trying to figure out where something "lives", I prefer Python's flat namespace to (for example) Java's com.company.baz.bar.foo hierarchy. > - 3rd party modules: There are many superb modules out there, some of which > really do have a "standard" character. Examples include PIL, numarray, > ElementTree, [wxPython - I know this is a hairy issue], I think the most important question for each of these is "is the module's release schedule at least as stable as Python's?". For many of these, I suspect the answer is "no". -- Tim Lesher <[EMAIL PROTECTED]> ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts on stdlib evolvement
On Jun 6, 2005, at 3:10 PM, Raymond Hettinger wrote: >>> If Fred were up for it, I think ElementTree would be a wonderful, >>> must-have addition. >>> > > > >> I might be missing fine details of the English language here >> (what does "to be up for something" mean?), however, I believe >> ElementTree is an unlikely addition to the standard library. >> > > Rewritten: > > If Fredrik Lundh were supportive of the idea, I think the Standard > Library would benefit greatly from incorporating ElementTree. > Personally I'd like to see the standard library get smaller rather than larger. There's a whole lot of bit rot in there, and since sys.path prefers the standard library over anything else it's a really huge pain to integrate patches on a faster release schedule than Python's while remaining sane at the same time. -bob ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts on stdlib evolvement
> > If Fred were up for it, I think ElementTree would be a wonderful, > > must-have addition. > I might be missing fine details of the English language here > (what does "to be up for something" mean?), however, I believe > ElementTree is an unlikely addition to the standard library. Rewritten: If Fredrik Lundh were supportive of the idea, I think the Standard Library would benefit greatly from incorporating ElementTree. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts on stdlib evolvement
Raymond Hettinger wrote: >> ElementTree, [wxPython - I know this is a hairy issue], > > > If Fred were up for it, I think ElementTree would be a wonderful, > must-have addition. I might be missing fine details of the English language here (what does "to be up for something" mean?), however, I believe ElementTree is an unlikely addition to the standard library. I also might be confusing people: AFAIK, ElementTree is Fredrik Lundh's work, not Fred Drake's, so it would be Fredrik who would need to contribute. My understanding is that Fredrik has indicated a dislike to contribute certain pieces of work, primarily out of fear to lose control. My experience with integrating his work without his explicit cooperation (specifically: sgmlop in PyXML) show that he won't object to somebody else doing such integration, but also won't try to cooperate afterwards. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts on stdlib evolvement
Skip Montanaro wrote:
> The main technical challenge seems to be
> backward compatibility. You need to support both flat ("import
> urllib") and
> packaged namespaces ("from www import urllib"), possibly within the
> same
> application. That is, postulating a www package, if I execute
>
> import urllib
> from www.urllib import urlopen
>
> the module-level code should only be executed once, and
>
> urlopen == urllib.urlopen
>
> should evaluate to True.
Unless
from __future__ import new_std_library
and we don't allow mixed use within the same module.
--eric
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts on stdlib evolvement
> "Barry" == Barry Warsaw <[EMAIL PROTECTED]> writes: Barry> On Mon, 2005-06-06 at 14:38, Skip Montanaro wrote: >> import urllib >> from www.urllib import urlopen >> >> the module-level code should only be executed once, and >> >> urlopen == urllib.urlopen >> >> should evaluate to True. Barry> Not to mention "urlopen is urllib.urlopen" Whoops, yeah. I was thinking in terms of module-level functions and classes, where I think == is sufficient: >>> import foo2 >>> foo2.f >>> g = foo2.f >>> reload(foo2) >>> foo2.f == g False Obviously, for data objects "is" is what you want. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts on stdlib evolvement
On Mon, 2005-06-06 at 14:38, Skip Montanaro wrote: > import urllib > from www.urllib import urlopen > > the module-level code should only be executed once, and > > urlopen == urllib.urlopen > > should evaluate to True. Not to mention "urlopen is urllib.urlopen" -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts on stdlib evolvement
Reinhold> - Flat namespace: Should we tend to a more hierarchic library
Reinhold> (e.g. inet.url, inet.http, inet.nntp)? This would increase
Reinhold> clarity when searching for a module.
We've talked about this before. The main technical challenge seems to be
backward compatibility. You need to support both flat ("import urllib") and
packaged namespaces ("from www import urllib"), possibly within the same
application. That is, postulating a www package, if I execute
import urllib
from www.urllib import urlopen
the module-level code should only be executed once, and
urlopen == urllib.urlopen
should evaluate to True.
Reinhold> - 3rd party modules: There are many superb modules out there,
Reinhold> some of which really do have a "standard"
Reinhold> character. Examples include PIL, numarray, ElementTree,
Reinhold> [wxPython - I know this is a hairy issue],
I think a good first step would be deciding on a package hierarchy. Third
party packages could plug in wherever they fit best.
Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts on stdlib evolvement
> - 3rd party modules: There are many superb modules out there, some of > which > really do have a "standard" character. Examples include PIL, numarray, > ElementTree, [wxPython - I know this is a hairy issue], If Fred were up for it, I think ElementTree would be a wonderful, must-have addition. Raymond Hettinger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Thoughts on stdlib evolvement
Hello, I am currently having some thoughts about the standard library, with regard to Python 2.5 and 3.0. Since I don't want to withhold them from you, here are they ;) - Flat namespace: Should we tend to a more hierarchic library (e.g. inet.url, inet.http, inet.nntp)? This would increase clarity when searching for a module. - 3rd party modules: There are many superb modules out there, some of which really do have a "standard" character. Examples include PIL, numarray, ElementTree, [wxPython - I know this is a hairy issue], - User interface: Tkinter may be "the" standard, but other Toolkits deserve notice, notably wxPython and PyGtk, the latter of which might be even easier to package. Reinhold, please don't hit me! -- Mail address is perfectly valid! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 rewrite complete
Paul Moore wrote:
> Unfortunately, I don't have a better naming suggestion (other than
> simply "template", which is probably too general to work), so I just
> raise this as something you should expect to need to clarify a few
> times...
PEP 346 used "statement_template" for the generator decorator. That
could be shortened to "stmt_template" without losing much in
readability ("stmt" being a fairly common abbreviation of "statement").
Avoiding the problem with the English meaning of the phrases "do
template" and "with template" was part of the reason for PEP 346's
choice of name (the other part being that the name tied in nicely with
that PEP's "user defined statement" terminology).
Cheers,
Nick.
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---
http://boredomandlaziness.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
