[Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
Currently, the naming section of PEP 8 doesn't say very much about what a leading underscore *means* in the Python standard library. I would like to add a new "Private interfaces" subsection under "Naming Conventions" to say the following: = Private interfaces Unless explicitly documented otherwise, a leading underscore on any name indicates that it is an internal implementation detail and backwards compatibility guarantees do not apply. It is strongly encouraged that private APIs (whether modules, classes, functions, attributes or other names) be clearly marked in this way, as many Python users rely on introspection to identify available functionality and may be mislead into believing an API without the leading underscore is in fact a public API with the standard backwards compatibility guarantees. All test modules are also considered private interfaces. Even though they typically lack the leading underscore, modules imported by another module are also considered an implementation detail. Other modules *should* not rely on indirect access to such modules unless they are an explicitly documented part of the API (such as ``os.path``). = Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.4 and Windows XP: just 45 days until EOL
Ben Finney writes: > "Stephen J. Turnbull" writes: > > > I don't see any good reason to take into account what Microsoft does > > or doesn't support. > > It seems you're advocating a position quite ad odds with > http://www.python.org/dev/peps/pep-0011/#id7>. Not at all. The first thing that the PEP says about unsupporting code is: Unsupporting platforms If a certain platform that currently has special code in it is deemed to be without Python users, What a vendor supports is only a heuristic. Existence of users comes first. Note that the policy says that some Windows platforms *will* be supported. It doesn't say others will be unsupported (except implicitly: 3 years after the last version of Visual Studio capable of building releases for that platform goes out of extended support, the build infrastructure will be removed). I don't see a good reason to change the PEP. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 14 July 2013 18:11, Nick Coghlan wrote: > Currently, the naming section of PEP 8 doesn't say very much about what a > leading underscore *means* in the Python standard library. > > I would like to add a new "Private interfaces" subsection under "Naming > Conventions" to say the following: > > = > Private interfaces > > Unless explicitly documented otherwise, a leading underscore on any name > indicates that it is an internal implementation detail and backwards > compatibility guarantees do not apply. It is strongly encouraged that > private APIs (whether modules, classes, functions, attributes or other > names) be clearly marked in this way, as many Python users rely on > introspection to identify available functionality and may be mislead into > believing an API without the leading underscore is in fact a public API > with the standard backwards compatibility guarantees. > > All test modules are also considered private interfaces. > > Even though they typically lack the leading underscore, modules imported > by another module are also considered an implementation detail. Other > modules *should* not rely on indirect access to such modules unless they > are an explicitly documented part of the API (such as ``os.path``). > = > Slight adjustment to the proposed wording to ensure completely undocumented modules are also considered private: = Private interfaces Unless explicitly documented otherwise, a leading underscore on any name indicates that it is an internal implementation detail and backwards compatibility guarantees do not apply. It is strongly encouraged that private APIs (whether modules, classes, functions, attributes or other names) be clearly marked in this way, as Python users may rely on introspection to identify available functionality and may be misled into believing an API without a leading underscore is in fact a public API with the standard backwards compatibility guarantees. Even when their names do not start with a leading underscore, all test modules and all modules that are not covered in the documentation are also considered private interfaces. Similarly, the specific modules and external APIs imported by a module are always considered an implementation detail. Other modules should not rely on indirect access to such imported interfaces unless they are an explicitly documented part of the containing module's API (such as ``os.path`` or a package ``__init__`` module exposing functionality from submodules). = -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PLY in stdlib (was cffi in stdlib)
On Sun, Jul 14, 2013 at 2:54 AM, Nick Coghlan wrote: > On 13 July 2013 23:26, David Beazley wrote: > >> I'm in favor of PLY going into stdlib with the caveat that there are some >> things about it that should probably be cleaned up and modernized. For >> instance, the method by which it writes the cached parsing tables needs to >> be cleaned up. I still think putting the LALR(1) generator code into a >> common library usable by both PLY/RPLY would be a useful thing to do. That >> code is really hairy and non-trivial to understand without something like >> the Dragon book nearby (and even then it's not easy). >> >> So, if I were to make any kind of proposal, I would say, make a standard >> library module for just the LALR(1) generator code. If the PLY interface >> is needed to add pycparser or cffi to the standard library, that can be >> added too, but as a separate module that uses the more general LALR(1) >> library. >> > > lrparsing is a more recent entry in the LR parsing stakes: > https://pypi.python.org/pypi/lrparsing (although, as Russell put it in > his PyCon AU lightning talk, if PLY had shown up ranked higher than 506 in > his PyPI search for "parser", he probably would have just used that: > http://pyvideo.org/video//sunday-lightning-talks at about 2:15) > > (I plan to bug Russell about putting that up on one of the DVCS hosting > sites next time I see him at BrisPy - for the moment, the source is > available through the tarball/sdist) > It seems a bit new compared to PLY's 15 years of existence to be considered in the running. Plus Russell would need to change the license from GPL. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PLY in stdlib (was cffi in stdlib)
On Jul 14, 2013, at 8:13 AM, Brett Cannon wrote: > > > > On Sun, Jul 14, 2013 at 2:54 AM, Nick Coghlan wrote: > On 13 July 2013 23:26, David Beazley wrote: > I'm in favor of PLY going into stdlib with the caveat that there are some > things about it that should probably be cleaned up and modernized. For > instance, the method by which it writes the cached parsing tables needs to be > cleaned up. I still think putting the LALR(1) generator code into a common > library usable by both PLY/RPLY would be a useful thing to do. That code is > really hairy and non-trivial to understand without something like the Dragon > book nearby (and even then it's not easy). > > So, if I were to make any kind of proposal, I would say, make a standard > library module for just the LALR(1) generator code. If the PLY interface is > needed to add pycparser or cffi to the standard library, that can be added > too, but as a separate module that uses the more general LALR(1) library. > > lrparsing is a more recent entry in the LR parsing > stakes:https://pypi.python.org/pypi/lrparsing (although, as Russell put it in > his PyCon AU lightning talk, if PLY had shown up ranked higher than 506 in > his PyPI search for "parser", he probably would have just used that: > http://pyvideo.org/video//sunday-lightning-talks at about 2:15) > > (I plan to bug Russell about putting that up on one of the DVCS hosting sites > next time I see him at BrisPy - for the moment, the source is available > through the tarball/sdist) > > It seems a bit new compared to PLY's 15 years of existence to be considered > in the running. Plus Russell would need to change the license from GPL. I honestly don't have any particular thoughts about PLY vs. other parser generators and the merits of their inclusion (or not) in the standard library. My impression has always been that the main interest in PLY was due to interest in seeing CFFI in the standard library. I'd say my main desire on the PLY side is that if it does go into the standard library, perhaps I could make it slightly less of mysterious black box and clean up a few bits. Cheers, Dave ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On Sun, Jul 14, 2013 at 7:09 AM, Nick Coghlan wrote: > On 14 July 2013 18:11, Nick Coghlan wrote: > >> Currently, the naming section of PEP 8 doesn't say very much about what a >> leading underscore *means* in the Python standard library. >> >> I would like to add a new "Private interfaces" subsection under "Naming >> Conventions" to say the following: >> >> = >> Private interfaces >> >> Unless explicitly documented otherwise, a leading underscore on any name >> indicates that it is an internal implementation detail and backwards >> compatibility guarantees do not apply. It is strongly encouraged that >> private APIs (whether modules, classes, functions, attributes or other >> names) be clearly marked in this way, as many Python users rely on >> introspection to identify available functionality and may be mislead into >> believing an API without the leading underscore is in fact a public API >> with the standard backwards compatibility guarantees. >> >> All test modules are also considered private interfaces. >> >> Even though they typically lack the leading underscore, modules imported >> by another module are also considered an implementation detail. Other >> modules *should* not rely on indirect access to such modules unless they >> are an explicitly documented part of the API (such as ``os.path``). >> = >> > > Slight adjustment to the proposed wording to ensure completely > undocumented modules are also considered private: > > = > Private interfaces > > Unless explicitly documented otherwise, a leading underscore on any name > indicates that it is an internal implementation detail and backwards > compatibility guarantees do not apply. It is strongly encouraged that > private APIs (whether modules, classes, functions, attributes or other > names) be clearly marked in this way, as Python users may rely on > introspection to identify available functionality and may be misled into > believing an API without a leading underscore is in fact a public API with > the standard backwards compatibility guarantees. > > Even when their names do not start with a leading underscore, all test > modules and all modules that are not covered in the documentation are also > considered private interfaces. > > Similarly, the specific modules and external APIs imported by a module are > always considered an implementation detail. Other modules should not rely > on indirect access to such imported interfaces unless they are an > explicitly documented part of the containing module's API (such as > ``os.path`` or a package ``__init__`` module exposing functionality from > submodules). > = > +1 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
+1 This is already how we've been behaving for years. On Sun, Jul 14, 2013 at 4:09 AM, Nick Coghlan wrote: > On 14 July 2013 18:11, Nick Coghlan wrote: > >> Currently, the naming section of PEP 8 doesn't say very much about what a >> leading underscore *means* in the Python standard library. >> >> I would like to add a new "Private interfaces" subsection under "Naming >> Conventions" to say the following: >> >> > Slight adjustment to the proposed wording to ensure completely > undocumented modules are also considered private: > > = > Private interfaces > > Unless explicitly documented otherwise, a leading underscore on any name > indicates that it is an internal implementation detail and backwards > compatibility guarantees do not apply. It is strongly encouraged that > private APIs (whether modules, classes, functions, attributes or other > names) be clearly marked in this way, > > as Python users may rely on introspection to identify available > functionality and may be misled into believing an API without a leading > underscore is in fact a public API with the standard backwards > compatibility guarantees. > While true, I'm not sure the last part of the sentence is necessary. Once we've established that a leading _ indicates something is private there isn't much point in explaining the various ways people might find them. I'm happy regardless of this bit being there. > Even when their names do not start with a leading underscore, all test > modules and all modules that are not covered in the documentation are also > considered private interfaces. > > Similarly, the specific modules and external APIs imported by a module are > always considered an implementation detail. Other modules should not rely > on indirect access to such imported interfaces unless they are an > explicitly documented part of the containing module's API (such as > ``os.path`` or a package ``__init__`` module exposing functionality from > submodules). > = > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 14/07/13 21:09, Nick Coghlan wrote: Slight adjustment to the proposed wording to ensure completely undocumented modules are also considered private: -1 on this adjustment. If somebody cannot be bothered writing a one-line doc string: "This module is private, don't touch." then they certainly shouldn't be allowed to use up a public name for a private module. I don't think we should be encouraging more private, undocumented modules. (Documentation is valuable even for private modules.) I'd go further, and say that no more private modules should be accepted for the std lib unless they have a leading underscore. I suppose for backwards compatibility reasons, we probably can't go through the std lib and rename private modules to make it clear they are private, but we don't have to accept new ones without the underscore. -- Steven ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 15Jul2013 09:48, Steven D'Aprano wrote: | On 14/07/13 21:09, Nick Coghlan wrote: | | >Slight adjustment to the proposed wording to ensure completely undocumented | >modules are also considered private: | | -1 on this adjustment. If somebody cannot be bothered writing a one-line doc string: | | "This module is private, don't touch." | | then they certainly shouldn't be allowed to use up a public name for a private module. I don't think we should be encouraging more private, undocumented modules. (Documentation is valuable even for private modules.) | | I'd go further, and say that no more private modules should be accepted for the std lib unless they have a leading underscore. I suppose for backwards compatibility reasons, we probably can't go through the std lib and rename private modules to make it clear they are private, but we don't have to accept new ones without the underscore. I disagree. A private module is a perfectly sane way to implement the internals of something, especially if it is subject to implementation change in the future. Clarification: is Nick classifying a module with docstrings by no content in the "modules" section of the Python doco as undocumented? That is what I would presume; I'd expect the code to be littered with docstrings anyway, but the module as a whole is not presented in the documentation and so should be private and not relied upon. Cheers, -- Cameron Simpson I sometimes wish that people would put a little more emphasis upon the observance of the law than they do upon its enforcement. - Calvin Coolidge ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Why is the return value of __contains__ coerced to boolean, but that of __lt__ and the like is not?
I'm curious why the return value of __contains__ is coerced to True or False, whereas the return value of "normal" comparison operators like __lt__ and the like are not. The latter return the value directly without forcing it to be True or False. This makes overriding __contains__ significantly less flexible, so I'm wondering why it's designed or implemented that way. (I believe it's the cmp_outcome() function in Python/ceval.c that does this: http://hg.python.org/cpython/file/db9fe49069ed/Python/ceval.c#l4545) For example, the "peewee" ORM overloads __lt__ and the like so it can map Python expressions to SQL. But it can't do this with the "in" operator due to the result of "x in y" always returning True or False in Python. So it (ab)uses the "<<" operator to do this instead (See the peewee docs at http://peewee.readthedocs.org/en/latest/peewee/querying.html#column-lookups ). I'm sure there's a good reason for why "in" is different here, but I can't see why right now. -Ben ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On Sun, Jul 14, 2013 at 8:01 PM, Cameron Simpson wrote: > On 15Jul2013 09:48, Steven D'Aprano wrote: > | On 14/07/13 21:09, Nick Coghlan wrote: > | > | >Slight adjustment to the proposed wording to ensure completely > undocumented > | >modules are also considered private: > | > | -1 on this adjustment. If somebody cannot be bothered writing a one-line > doc string: > | > | "This module is private, don't touch." > | > | then they certainly shouldn't be allowed to use up a public name for a > private module. I don't think we should be encouraging more private, > undocumented modules. (Documentation is valuable even for private modules.) > | > | I'd go further, and say that no more private modules should be accepted > for the std lib unless they have a leading underscore. I suppose for > backwards compatibility reasons, we probably can't go through the std lib > and rename private modules to make it clear they are private, but we don't > have to accept new ones without the underscore. > > I disagree. > > A private module is a perfectly sane way to implement the internals > of something, especially if it is subject to implementation change > in the future. > > Clarification: is Nick classifying a module with docstrings by no > content in the "modules" section of the Python doco as undocumented? > Yes. This has nothing to do with docstrings, just the official documentation at docs.python.org. > That is what I would presume; I'd expect the code to be littered > with docstrings anyway, but the module as a whole is not presented > in the documentation and so should be private and not relied upon. > Exactly. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is the return value of __contains__ coerced to boolean, but that of __lt__ and the like is not?
Oh, just to show what I mean here with some code (same in both Python 2.x
and 3.x):
-
from __future__ import print_function
class C(object):
def __contains__(self, x):
print('__contains__({0!r})'.format(x))
return x
def __lt__(self, x):
print('__lt__({0!r})'.format(x))
return x
c = C()
print(42 in c)
print(0 in c)
print(c < 42)
print(c < 0)
-
This prints the following:
__contains__(42)
True
__contains__(0)
False
__lt__(42)
42
__lt__(0)
0
Whereas I'd kinda expect it to print:
__contains__(42)
42
__contains__(0)
0
__lt__(42)
42
__lt__(0)
0
-Ben
On Mon, Jul 15, 2013 at 12:21 PM, Ben Hoyt wrote:
> I'm curious why the return value of __contains__ is coerced to True or
> False, whereas the return value of "normal" comparison operators like
> __lt__ and the like are not. The latter return the value directly without
> forcing it to be True or False.
>
> This makes overriding __contains__ significantly less flexible, so I'm
> wondering why it's designed or implemented that way. (I believe it's the
> cmp_outcome() function in Python/ceval.c that does this:
> http://hg.python.org/cpython/file/db9fe49069ed/Python/ceval.c#l4545)
>
> For example, the "peewee" ORM overloads __lt__ and the like so it can map
> Python expressions to SQL. But it can't do this with the "in" operator due
> to the result of "x in y" always returning True or False in Python. So it
> (ab)uses the "<<" operator to do this instead (See the peewee docs at
> http://peewee.readthedocs.org/en/latest/peewee/querying.html#column-lookups
> ).
>
> I'm sure there's a good reason for why "in" is different here, but I can't
> see why right now.
>
> -Ben
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 15 July 2013 09:48, Steven D'Aprano wrote: > On 14/07/13 21:09, Nick Coghlan wrote: > >> Slight adjustment to the proposed wording to ensure completely >> undocumented >> modules are also considered private: > > > -1 on this adjustment. If somebody cannot be bothered writing a one-line doc > string: > > "This module is private, don't touch." > > then they certainly shouldn't be allowed to use up a public name for a > private module. I don't think we should be encouraging more private, > undocumented modules. (Documentation is valuable even for private modules.) For context, this arose when I checked PEP 8 after the point was raised on distutils-sig that pip uses a public name for its implementation module (also pip, same as the CLI), but officially exposes no stable public programmatic API. At the moment, according to what little PEP 8 has to say about public vs private interfaces, that means a bundled pip would represent a new public API, despite the fact that the only documented interface for pip is the CLI, not the Python module API. Since we've been burned by people assuming private APIs are public in the past, I figured it made sense to get the actual standards we follow in practice documented properly, *before* anyone further gets the wrong idea from "import pip; help(pip)". Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is the return value of __contains__ coerced to boolean, but that of __lt__ and the like is not?
On 15 July 2013 10:21, Ben Hoyt wrote: > I'm sure there's a good reason for why "in" is different here, but I can't > see why right now. It depends on what you mean by "good reason" - PEP 207 (which is what allows arbitrary objects to be returned from comparison operations) was entirely about replacing __cmp__ with the rich comparison methods, it doesn't mention __contains__ at all. At this point the main limitations are backwards compatibility (having existing containment tests suddenly start returning anything other than True or False would be problematic), along with the signature of CPython's sq_contains slot (it returns an integer rather than a PyObject pointer). Accordingly, to convert containment testing to a rich comparison operation would require a new protocol. That said, there is potential value in redefining containment in terms of a symmetric protocol (rather than the current only-controlled-by-the-container behaviour), so such a PEP may be worth writing. (it would initially be a topic for python-ideas rather than python-dev, though) Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 7/14/2013 7:09 AM, Nick Coghlan wrote: Slight adjustment to the proposed wording to ensure completely undocumented modules are also considered private: = Private interfaces Unless explicitly documented otherwise, a leading underscore on any name indicates that it is an internal implementation detail and backwards compatibility guarantees do not apply. It is strongly encouraged that private APIs (whether modules, classes, functions, attributes or other names) be clearly marked in this way, as Python users may rely on introspection to identify available functionality and may be misled into believing an API without a leading underscore is in fact a public API with the standard backwards compatibility guarantees. Even when their names do not start with a leading underscore, all test modules and all modules that are not covered in the documentation are also considered private interfaces. I was going to suggest adding 'and most idlelib ' between 'test' and 'modules', but the broader addition covers idlelib, which is not mentioned yet in the docs, even in the (unindexed) Idle chapter. When it is, I will try to remember to make explicit which names and interfaces are public and that the rest are private. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is the return value of __contains__ coerced to boolean, but that of __lt__ and the like is not?
Thanks, Nick -- that's helpful info. Writing such a PEP is a nice idea, but I think it'd be beyond me (I'm not familiar enough with CPython internals, protocols, etc). Can you explain what you mean by "symmetric protocol rather than the current only-controlled-by-the-container behaviour"? -Ben On Mon, Jul 15, 2013 at 12:45 PM, Nick Coghlan wrote: > On 15 July 2013 10:21, Ben Hoyt wrote: > > I'm sure there's a good reason for why "in" is different here, but I > can't > > see why right now. > > It depends on what you mean by "good reason" - PEP 207 (which is what > allows arbitrary objects to be returned from comparison operations) > was entirely about replacing __cmp__ with the rich comparison methods, > it doesn't mention __contains__ at all. > > At this point the main limitations are backwards compatibility (having > existing containment tests suddenly start returning anything other > than True or False would be problematic), along with the signature of > CPython's sq_contains slot (it returns an integer rather than a > PyObject pointer). > > Accordingly, to convert containment testing to a rich comparison > operation would require a new protocol. That said, there is potential > value in redefining containment in terms of a symmetric protocol > (rather than the current only-controlled-by-the-container behaviour), > so such a PEP may be worth writing. (it would initially be a topic for > python-ideas rather than python-dev, though) > > Cheers, > Nick. > > -- > Nick Coghlan | [email protected] | Brisbane, Australia > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On Mon, Jul 15, 2013 at 10:30:02AM +1000, Nick Coghlan wrote: > On 15 July 2013 09:48, Steven D'Aprano wrote: > > On 14/07/13 21:09, Nick Coghlan wrote: > > > >> Slight adjustment to the proposed wording to ensure completely > >> undocumented > >> modules are also considered private: > > > > > > -1 on this adjustment. If somebody cannot be bothered writing a one-line doc > > string: > > > > "This module is private, don't touch." > > > > then they certainly shouldn't be allowed to use up a public name for a > > private module. I don't think we should be encouraging more private, > > undocumented modules. (Documentation is valuable even for private modules.) > > For context, this arose when I checked PEP 8 after the point was > raised on distutils-sig that pip uses a public name for its > implementation module (also pip, same as the CLI), but officially > exposes no stable public programmatic API. I don't think we should bless this as official policy, certainly not for the standard library. We can't tell external modules what to do, and possibly there's nothing we can do about existing undocumented private modules, short of renaming them, but I think that at least for new additions to the std lib module names should follow the same single-underscore convention as functions, classes etc. A single leading underscore introduces the private namespace, everything else is public. I've seen too much undocumented public code to ever assume that lack of documentation implies it is private. > At the moment, according to what little PEP 8 has to say about public > vs private interfaces, that means a bundled pip would represent a new > public API, despite the fact that the only documented interface for > pip is the CLI, not the Python module API. > > Since we've been burned by people assuming private APIs are public in > the past, I figured it made sense to get the actual standards we > follow in practice documented properly, *before* anyone further gets > the wrong idea from "import pip; help(pip)". I can tell you that people will do that regardless of what PEP 8 has to say about "not documented == private". Especially as worded. I'm sure I won't be the only person to reason that if a module has a docstring, it is therefore documented. It takes one character to explicitly document a module as private. That's much better than expecting people to treat their failure to find documentation as an implicit warning. -- Steven ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On Mon, Jul 15, 2013 at 10:01:17AM +1000, Cameron Simpson wrote: > On 15Jul2013 09:48, Steven D'Aprano wrote: > | I'd go further, and say that no more private modules should be > | accepted for the std lib unless they have a leading underscore. I > | suppose for backwards compatibility reasons, we probably can't go > | through the std lib and rename private modules to make it clear they > | are private, but we don't have to accept new ones without the > | underscore. > > I disagree. > > A private module is a perfectly sane way to implement the internals > of something, especially if it is subject to implementation change > in the future. Of course private modules are sane. I never suggested "no new private modules at all". But putting them in the same namespace as public modules is not, just to save a leading underscore in the file name. You don't even have to use the underscore in your own code: import _stuff as stuff is allowed, and doesn't make _stuff.py public since imported modules are considered implementation details by default. -- Steven ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is the return value of __contains__ coerced to boolean, but that of __lt__ and the like is not?
On Mon, Jul 15, 2013 at 03:34:08PM +1200, Ben Hoyt wrote: > Thanks, Nick -- that's helpful info. Writing such a PEP is a nice idea, but > I think it'd be beyond me (I'm not familiar enough with CPython internals, > protocols, etc). > > Can you explain what you mean by "symmetric protocol rather than the > current only-controlled-by-the-container behaviour"? Most operators can be controlled by either the left-hand or right-hand operand. For example, x + y can end up calling either x.__add__(y) or y.__radd_(x). The `in` operator is an exception, it only ever calls the container: x in y => y.__contains__(x) but never x.__contained_by__(y) -- Steven ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 15 Jul 2013 13:43, "Steven D'Aprano" wrote: > > On Mon, Jul 15, 2013 at 10:30:02AM +1000, Nick Coghlan wrote: > > On 15 July 2013 09:48, Steven D'Aprano wrote: > > > On 14/07/13 21:09, Nick Coghlan wrote: > > > > > >> Slight adjustment to the proposed wording to ensure completely > > >> undocumented > > >> modules are also considered private: > > > > > > > > > -1 on this adjustment. If somebody cannot be bothered writing a one-line doc > > > string: > > > > > > "This module is private, don't touch." > > > > > > then they certainly shouldn't be allowed to use up a public name for a > > > private module. I don't think we should be encouraging more private, > > > undocumented modules. (Documentation is valuable even for private modules.) > > > > For context, this arose when I checked PEP 8 after the point was > > raised on distutils-sig that pip uses a public name for its > > implementation module (also pip, same as the CLI), but officially > > exposes no stable public programmatic API. > > I don't think we should bless this as official policy, certainly not for > the standard library. We can't tell external modules what to do, and > possibly there's nothing we can do about existing undocumented private > modules, short of renaming them, but I think that at least for new > additions to the std lib module names should follow the same > single-underscore convention as functions, classes etc. A single leading > underscore introduces the private namespace, everything else is public. > > I've seen too much undocumented public code to ever assume that lack of > documentation implies it is private. > > > > At the moment, according to what little PEP 8 has to say about public > > vs private interfaces, that means a bundled pip would represent a new > > public API, despite the fact that the only documented interface for > > pip is the CLI, not the Python module API. > > > > Since we've been burned by people assuming private APIs are public in > > the past, I figured it made sense to get the actual standards we > > follow in practice documented properly, *before* anyone further gets > > the wrong idea from "import pip; help(pip)". > > I can tell you that people will do that regardless of what PEP 8 has to > say about "not documented == private". Especially as worded. I'm sure I > won't be the only person to reason that if a module has a docstring, it > is therefore documented. > > It takes one character to explicitly document a module as private. > That's much better than expecting people to treat their failure to > find documentation as an implicit warning. We're considering bundling the pip CLI for the convenience of educators and beginners, not adding their implementation module to the standard library. That means we need a bright shining line for what constitutes "in the standard library". Yes, it's better if people properly use a leading underscore, but the fact is many don't (including idlelib, as Terry noted) and it becomes a disruptive change to add it later if people don't start with it. The pip and IDLE developers have better things to be doing than changing the names of undocumented APIs just because the application is distributed along with CPython. Cheers, Nick. > > > > -- > Steven > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 15 Jul 2013 13:46, "Steven D'Aprano" wrote: > > On Mon, Jul 15, 2013 at 10:01:17AM +1000, Cameron Simpson wrote: > > On 15Jul2013 09:48, Steven D'Aprano wrote: > > > | I'd go further, and say that no more private modules should be > > | accepted for the std lib unless they have a leading underscore. I > > | suppose for backwards compatibility reasons, we probably can't go > > | through the std lib and rename private modules to make it clear they > > | are private, but we don't have to accept new ones without the > > | underscore. > > > > I disagree. > > > > A private module is a perfectly sane way to implement the internals > > of something, especially if it is subject to implementation change > > in the future. > > Of course private modules are sane. I never suggested "no new private > modules at all". But putting them in the same namespace as public > modules is not, just to save a leading underscore in the file name. It's not to save a leading underscore - it's to avoid renaming existing packages like pip and idlelib. Cheers, Nick. > > You don't even have to use the underscore in your own code: > > import _stuff as stuff > > is allowed, and doesn't make _stuff.py public since imported modules are > considered implementation details by default. > > > -- > Steven > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 15 July 2013 02:15, Gregory P. Smith wrote: > On Sun, Jul 14, 2013 at 4:09 AM, Nick Coghlan wrote: >> as Python users may rely on introspection to identify available >> functionality and may be misled into believing an API without a leading >> underscore is in fact a public API with the standard backwards compatibility >> guarantees. > > > While true, I'm not sure the last part of the sentence is necessary. Once > we've established that a leading _ indicates something is private there > isn't much point in explaining the various ways people might find them. I'm > happy regardless of this bit being there. You'd be surprised how many non-core devs react with astonishment when I suggest that not documenting something isn't enough to avoid having users consider it a supported public API - they usually get it after I point out how far you can usually get just by using dir() and help() to play with a new library at the interactive prompt instead of looking at out-of-band docs. I figure including the second part will help prevent some "But why?" reactions in the future. I think Steven has a reasonable point about being clearer that an explicit leading underscore is the preferred solution for any new private modules, though, so here's an updated proposal: = Private interfaces Unless explicitly documented otherwise, a leading underscore on any name indicates that it is an internal implementation detail and any backwards compatibility guarantees do not apply. It is strongly encouraged that private APIs (whether modules, classes, functions, attributes or other names) be clearly marked in this way, as Python users may rely on introspection to identify available functionality and may be misled into believing an API without a leading underscore is in fact a public API with the standard backwards compatibility guarantees. While the explicit use of a leading underscore in the names of private modules is preferred, all test modules and all modules that are not explicitly covered in the documentation are also considered private interfaces, even when their names do not start with a leading underscore and even if they include a module level documentation string. This includes submodules of packages that are documented as if they were a single module. Similarly, the specific modules and external APIs imported by a module are always considered an implementation detail. Other modules should not rely on indirect access to such imported interfaces unless they are an explicitly documented part of the containing module's API (such as ``os.path`` or a package ``__init__`` module that exposes functionality from submodules). = -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On Mon, Jul 15, 2013 at 6:17 AM, Nick Coghlan wrote: > = > Private interfaces > > Unless explicitly documented otherwise, a leading underscore on any > name indicates that it is an internal implementation detail and any > backwards compatibility guarantees do not apply. It is strongly > encouraged that private APIs (whether modules, classes, functions, > attributes or other names) be clearly marked in this way, as Python > users may rely on introspection to identify available functionality > and may be misled into believing an API without a leading underscore > is in fact a public API with the standard backwards compatibility > guarantees. Very good. > While the explicit use of a leading underscore in the names of private > modules is preferred, all test modules and all modules that are not > explicitly covered in the documentation are also considered private > interfaces, even when their names do not start with a leading > underscore and even if they include a module level documentation > string. This includes submodules of packages that are documented as if > they were a single module. But wait, aren't this about how to use other peoples modules, more than a style guide of how to write your own modules? I don't think this belongs in PEP 8 at all. //Lennart ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
Nick Coghlan writes:
>> Of course private modules are sane. I never suggested "no new private
>> modules at all". But putting them in the same namespace as public
>> modules is not, just to save a leading underscore in the file name.
> It's not to save a leading underscore - it's to avoid renaming
> existing packages like pip and idlelib.
"existing" is not a subset of "new." Am I missing something?
What Steven is suggesting is that there are two kinds of private
package in the stdlib: those that can conveniently be given names that
indicate that they are private (generally, "new" packages), and those
that can't ("existing" packages).
For the latter IIRC he suggested adding a line at the top of the module
documentation "This module is *private*, and you cannot depend on API
compatibility with future or past releases of this module. The public
API is in ... [alternatively, There is no public API]."
I really don't see how one can object to this suggestion, given an
appropriate definition of "convenient enough to get a _name".
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PLY in stdlib (was cffi in stdlib)
On Jul 14, 2013, at 6:32 AM, David Beazley wrote: > I honestly don't have any particular thoughts about PLY vs. other parser > generators and the merits of their inclusion (or not) in the standard > library. I would love to have PLY in the standard library. It would open up a whole new world to some users and be the basis for tool generation for others. Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Doubt about closure/function generation in bytecode
2013/7/12 Andrea Griffini : > Is there any semantic difference between > > BUILD_TUPLE 0 > LOAD_CONST > MAKE_CLOSURE 0 > > and > > LOAD_CONST > MAKE_FUNCTION 0 > > ? > > In other words is there any difference between a function and a closure that > doesn't capture anything? Is it just a speed optimization for a common case? It probably has more to do with the fact that nested scopes with added much later in Python's history than functions. > -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
