subject:"\[Python\-Dev\] Tweaking PEP 8 guidelines for use of leading underscores"

On 16 July 2013 16:01, Chris McDonough chr...@plope.com wrote:
 Well, I know it's arguing with the umpire at this point, but PEP8
 prescriptionism (great word by the way!) is de rigeur these days.  It's
 a python-dev perogative to ignore that, but it has consequences that
 reverberate further than this maillist.

Yeah, we should probably be a bit more vocal in pointing out that PEP
8 is something to be applied judiciously, *not* treated as sacred
writ, never to be questioned. It's very tempting to just sigh and
ignore it, instead of explicitly pointing out the problems with taking
it too literally for people's personal projects (or, worse, using it
to abuse developers of projects that long predate the recommendations
in more recent versions) :P

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-16 Thread Terry Reedy


On 7/15/2013 11:11 PM, Nick Coghlan wrote:


I'll look into adding some stronger wording at the top making it clear
that while PEP 8 is a useful starting point and a good default if a
project doesn't have a defined style guide of it's own, it is *not*
the be-all-and-end-all for Python style guides. Treating it as such as
an abuse of the PEP, pure and simple.


How about (re)naming it
Style Guide for Python Standard Library Code

Without 'Standard Library' in the title, I can see how some people might 
think it meant to apply to all code. I do not think we need to worry 
about the above being too narrow.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-16 Thread Richard Oudkerk


On 16/07/2013 6:44am, Nick Coghlan wrote:

Clarifying what constitutes an internal interface in a way that
doesn't require renaming anything is a necessary prerequisite for
bundling or bootstrapping the pip CLI in Python 3.4 (as pip exposes
its internal implemetnation API as import pip rather than import
_pip and renaming it would lead to a lot of pointless code churn).
Without that concern, the topic never would have come up.


BTW, how does the use of __all__ effect things?  Somewhere I got the 
idea that if a module uses __all__ then anything not listed is internal. 
 I take it that is wrong?


--
Richard

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On 16 July 2013 18:41, Terry Reedy tjre...@udel.edu wrote:
 On 7/15/2013 11:11 PM, Nick Coghlan wrote:

 I'll look into adding some stronger wording at the top making it clear
 that while PEP 8 is a useful starting point and a good default if a
 project doesn't have a defined style guide of it's own, it is *not*
 the be-all-and-end-all for Python style guides. Treating it as such as
 an abuse of the PEP, pure and simple.


 How about (re)naming it
 Style Guide for Python Standard Library Code

 Without 'Standard Library' in the title, I can see how some people might
 think it meant to apply to all code. I do not think we need to worry about
 the above being too narrow.

I reread the whole thing today - it turns out we *have* tweaked it
over time to account for other people using it for their projects as
well. The one genuinely standard library specific element still in it
(the prohibition on the use of function annotations) is explicitly
qualified as only applying to the standard library and not anything
else. So I think we need to continue supporting that use case.

I did find it interesting that we *don't* explicitly advise against
the use of import * for anything other than optional accelerator
modules or republishing internal interfaces through a public API, even
though we advice against the practice in the tutorial. Perhaps another
oversight worth correcting?

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On 16 July 2013 20:28, Richard Oudkerk shibt...@gmail.com wrote:
 On 16/07/2013 6:44am, Nick Coghlan wrote:

 Clarifying what constitutes an internal interface in a way that
 doesn't require renaming anything is a necessary prerequisite for
 bundling or bootstrapping the pip CLI in Python 3.4 (as pip exposes
 its internal implemetnation API as import pip rather than import
 _pip and renaming it would lead to a lot of pointless code churn).
 Without that concern, the topic never would have come up.


 BTW, how does the use of __all__ effect things?  Somewhere I got the idea
 that if a module uses __all__ then anything not listed is internal.  I take
 it that is wrong?

Gah, you're right. I did consider that, then forgot to include it in
the proposed text.

Rather than throwing more versions of the text at the list, I'll
create a tracker issue and post a proposed patch for PEP 8.

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On 16 July 2013 21:10, Nick Coghlan ncogh...@gmail.com wrote:
 On 16 July 2013 20:28, Richard Oudkerk shibt...@gmail.com wrote:
 On 16/07/2013 6:44am, Nick Coghlan wrote:

 Clarifying what constitutes an internal interface in a way that
 doesn't require renaming anything is a necessary prerequisite for
 bundling or bootstrapping the pip CLI in Python 3.4 (as pip exposes
 its internal implemetnation API as import pip rather than import
 _pip and renaming it would lead to a lot of pointless code churn).
 Without that concern, the topic never would have come up.


 BTW, how does the use of __all__ effect things?  Somewhere I got the idea
 that if a module uses __all__ then anything not listed is internal.  I take
 it that is wrong?

 Gah, you're right. I did consider that, then forgot to include it in
 the proposed text.

 Rather than throwing more versions of the text at the list, I'll
 create a tracker issue and post a proposed patch for PEP 8.

Tracker issue: http://bugs.python.org/issue18472

When I went to write this in context, I realised it made more sense to
focus on defining what a *public* interface was, and how that differed
from an internal interface. WIth that approach, the natural flow of
the text was:

* the public interface is preferentially defined by the documentation
* __all__ is the main programmatic interface
* leading underscores are then mostly a helper that makes it easier to
remember which is which while actually *working* on the code (but
still a good practice)

I also spotted a few other oddities as I went through. The obviously
wrong ones I just fixed, but there were a couple relating to
exceptions that seem unnecessary or ill-advised given PEP 352 and
3151, plus the one about absolute vs explicit relative imports that I
started a separate thread for.

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-16 Thread Steven D'Aprano


On 16/07/13 20:28, Richard Oudkerk wrote:

On 16/07/2013 6:44am, Nick Coghlan wrote:

Clarifying what constitutes an internal interface in a way that
doesn't require renaming anything is a necessary prerequisite for
bundling or bootstrapping the pip CLI in Python 3.4 (as pip exposes
its internal implemetnation API as import pip rather than import
_pip and renaming it would lead to a lot of pointless code churn).
Without that concern, the topic never would have come up.


BTW, how does the use of __all__ effect things?  Somewhere I got the idea that 
if a module uses __all__ then anything not listed is internal.  I take it that 
is wrong?



That is not how I interpret __all__. In the absence of any explicit 
documentation, I interpret __all__ as nothing more than a list of names which 
wildcard imports will bring in, without necessarily meaning that other names 
are private. For example, I might have a module explicitly designed for 
wildcard imports at the interactive interpreter:

from module import *

brings in the functions which I expect will be useful interactively, not 
necessarily the entire public API.

For example, pkgutil includes classes with single-underscore methods, which I take as 
private. It also has a function simplegeneric, which is undocumented and not listed in 
__all__. In in the absence of even a comment saying Don't use this, I take it 
as an oversight, not policy that simplegeneric is private.



--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-16 Thread R. David Murray

On Tue, 16 Jul 2013 23:19:21 +1000, Steven D'Aprano st...@pearwood.info wrote:
 On 16/07/13 20:28, Richard Oudkerk wrote:
  On 16/07/2013 6:44am, Nick Coghlan wrote:
  Clarifying what constitutes an internal interface in a way that
  doesn't require renaming anything is a necessary prerequisite for
  bundling or bootstrapping the pip CLI in Python 3.4 (as pip exposes
  its internal implemetnation API as import pip rather than import
  _pip and renaming it would lead to a lot of pointless code churn).
  Without that concern, the topic never would have come up.
 
  BTW, how does the use of __all__ effect things?  Somewhere I got the idea 
  that if a module uses __all__ then anything not listed is internal.  I take 
  it that is wrong?
 
 
 That is not how I interpret __all__. In the absence of any explicit 
 documentation, I interpret __all__ as nothing more than a list of names which 
 wildcard imports will bring in, without necessarily meaning that other names 
 are private. For example, I might have a module explicitly designed for 
 wildcard imports at the interactive interpreter:
 
 from module import *
 
 brings in the functions which I expect will be useful interactively, not 
 necessarily the entire public API.
 
 For example, pkgutil includes classes with single-underscore methods, which I 
 take as private. It also has a function simplegeneric, which is undocumented 
 and not listed in __all__. In in the absence of even a comment saying Don't 
 use this, I take it as an oversight, not policy that simplegeneric is 
 private.

I think you'd be wrong about that, though.  simplegeneric should really be
treated as private.  I'm speaking here not about the general principle of
the thing, but about my understanding of simplegeneric's specific history.

--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On 16 July 2013 23:39, R. David Murray rdmur...@bitdance.com wrote:
 On Tue, 16 Jul 2013 23:19:21 +1000, Steven D'Aprano st...@pearwood.info 
 wrote:
 On 16/07/13 20:28, Richard Oudkerk wrote:
  On 16/07/2013 6:44am, Nick Coghlan wrote:
  Clarifying what constitutes an internal interface in a way that
  doesn't require renaming anything is a necessary prerequisite for
  bundling or bootstrapping the pip CLI in Python 3.4 (as pip exposes
  its internal implemetnation API as import pip rather than import
  _pip and renaming it would lead to a lot of pointless code churn).
  Without that concern, the topic never would have come up.
 
  BTW, how does the use of __all__ effect things?  Somewhere I got the idea 
  that if a module uses __all__ then anything not listed is internal.  I 
  take it that is wrong?


 That is not how I interpret __all__. In the absence of any explicit 
 documentation, I interpret __all__ as nothing more than a list of names 
 which wildcard imports will bring in, without necessarily meaning that other 
 names are private. For example, I might have a module explicitly designed 
 for wildcard imports at the interactive interpreter:

 from module import *

 brings in the functions which I expect will be useful interactively, not 
 necessarily the entire public API.

 For example, pkgutil includes classes with single-underscore methods, which 
 I take as private. It also has a function simplegeneric, which is 
 undocumented and not listed in __all__. In in the absence of even a comment 
 saying Don't use this, I take it as an oversight, not policy that 
 simplegeneric is private.

 I think you'd be wrong about that, though.  simplegeneric should really be
 treated as private.  I'm speaking here not about the general principle of
 the thing, but about my understanding of simplegeneric's specific history.

And, indeed, you're correct (the issue that eventually became the
functools.singledispatch PEP started life with a title like move
simplegeneric to functools and make it public).

For the general case, the patch I posted to the issue tracker sets the
precedence as docs - __all__ - leading underscore. If a module has
public APIs that aren't in __all__, it should cover them in the docs,
otherwise people should assume they're private. It's OK if there are
exceptions to that general rule - there's a reason PEP 8 has the great
big qualifier pointing out that it isn't universally applicable (even
if we might sometimes wish otherwise).

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On Jul 15, 2013, at 11:02 PM, Chris McDonough wrote:

Given that misunderstanding, is there a way to divorce stdlib-centric
guidelines like the one being discussed now from PEP8-ness?  I don't
think any amount of marketing effort or reasoned explanation is going to
separate PEP8 from correct code in people's minds at this point.

The standard seems to be if the pep8 program complains, I forward that
upstream, so it's probably most effective to discuss with them.  OTOH, we've
had to clarify PEP 8 in order to get the pep8 developers to adjust their tool,
most recently e.g. the closing brace/bracket/paren for multiline collections.

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On Jul 16, 2013, at 11:28 AM, Richard Oudkerk wrote:

BTW, how does the use of __all__ effect things?  Somewhere I got the idea
that if a module uses __all__ then anything not listed is internal.  I take
it that is wrong?

Purely technically, __all__ is there to affect how from-import-* works.  I
personally think it's a good idea to include all your public names, and none
of your non-public names, in __all__, but it's not always easy to keep
up-to-date.  pyflakes has the nice benefit of complaining when something is
named in __all__ that doesn't exist in the module, but that's only one part of
the keeping-things-up-to-date problem.

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On Jul 16, 2013, at 11:48 PM, Nick Coghlan wrote:

For the general case, the patch I posted to the issue tracker sets the
precedence as docs - __all__ - leading underscore.

+1

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On Jul 16, 2013, at 04:10 PM, Nick Coghlan wrote:

Yeah, we should probably be a bit more vocal in pointing out that PEP
8 is something to be applied judiciously, *not* treated as sacred
writ, never to be questioned. It's very tempting to just sigh and
ignore it, instead of explicitly pointing out the problems with taking
it too literally for people's personal projects (or, worse, using it
to abuse developers of projects that long predate the recommendations
in more recent versions) :P

In my experience, most PEP 8 violation bugs that I see for various upstreams
(mine and others) comes from running the pep8 tool over the code.  A lot of
upstreams do this as part of their normal QA (e.g. you can set up unittests
that run pep8, pyflakes, and other linters).  Those will generally be fine
because they're already committed to conformance to pep8's notion of what PEP
8 says wink[1].

In other cases, Someone Out There runs pep8 over your code, and files bugs on
all the complaints that get produced.  That can definitely lead to churn and
waste, especially on big, established code bases.

-Barry

[1] which I do not always agree with anyway, as evidenced by the recent
discussion on closing paren indentation for multi-line collections.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-16 Thread Terry Reedy


On 7/16/2013 9:39 AM, R. David Murray wrote:

On Tue, 16 Jul 2013 23:19:21 +1000, Steven D'Aprano st...@pearwood.info wrote:



For example, pkgutil includes classes with single-underscore methods, which I take as 
private. It also has a function simplegeneric, which is undocumented and not listed in 
__all__. In in the absence of even a comment saying Don't use this, I take it 
as an oversight, not policy that simplegeneric is private.


I think you'd be wrong about that, though.  simplegeneric should really be
treated as private.  I'm speaking here not about the general principle of
the thing, but about my understanding of simplegeneric's specific history.


I think Steven (valid) point is Why not, then, say it is internal 
either in docs or name?-- which in this case would be in the docs.



--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-16 Thread Tres Seaver

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 07/16/2013 07:09 AM, Nick Coghlan wrote:

 I did find it interesting that we *don't* explicitly advise against 
 the use of import * for anything other than optional accelerator 
 modules or republishing internal interfaces through a public API,
 even though we advice against the practice in the tutorial. Perhaps
 another oversight worth correcting?

+1.  'from foo import *' (from any source other than another module in
the same package) is a code stench far worse than anything else PEP8
proscribes.


Tres.
- -- 
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEARECAAYFAlHl2GsACgkQ+gerLs4ltQ6+1QCgmu5k6p5otzPxvzGh5mA1Ch7t
2f0AoK2a0/m144bnIwBFLLuY9l2bdWMN
=878p
-END PGP SIGNATURE-

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On Jul 16, 2013, at 07:34 PM, Tres Seaver wrote:

On 07/16/2013 07:09 AM, Nick Coghlan wrote:

 I did find it interesting that we *don't* explicitly advise against 
 the use of import * for anything other than optional accelerator 
 modules or republishing internal interfaces through a public API,
 even though we advice against the practice in the tutorial. Perhaps
 another oversight worth correcting?

+1.  'from foo import *' (from any source other than another module in
the same package) is a code stench far worse than anything else PEP8
proscribes.

Maybe we should disable it for anything that isn't at the interactive prompt
0.5 wink.

- -Barry
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.20 (GNU/Linux)

iQIcBAEBCAAGBQJR5e5VAAoJEBJutWOnSwa/SK4P/22KvWNRQ7sjDWJ+jebgpXyD
mkUeO4r905uqp5Bvedlyj4GoFhdFPQoeQEGgQeNwiMkFxCGJDKiY1XoJAgqj2zOE
vUgoyLE5YSAnz8xG/Ib3ZZPXhDJY14gu/czbX/cc83Q+cQJd0+RpnhUt5jQkxAYR
k6xXF7hwTHaTYlqzdTDQUtimg21AUb0lgI5XqCMGL4aac4H69JiN+3ydI3BQxqxY
MXe4h4rzDxnaBnekHYCouM1jsN6CsLeIHafJR8W4Zgm9vuxZYIha+kAQd+tILFXG
d/0hpvLIGP+ubNPIvS1Ay+aYsH7PiL4keCoP0pi+llMTQv+W5wDeCumLpvXjaEFj
46c7HaRaxbBqpactsG98rZOtmBwSgEwzzPSQak+sWHDCX4RFOS3qDh3WhtY0Lw7n
NdEkJ0f+6bEt2ot+emDpth/SAxgOEUjNOhZ8/glYtvZcz2wBWRGkCAIEwSJtP1VZ
YgRHFsoUUbS6KoI9cZWhwynmIVQf4Wn9a2Zx5/YUbHIM2mY1s60MxcOn3KgPpEU4
Zjp/gyPHsoIORrZU2Q2pwzdWEmSzorZBP/NqRqlhwBTexKdfqyRFztsFMVcnLPDv
Cl0t04BGXH8/0fZ+peEOccqnuPsHumr9VqmhMCcLChRPRQyBD9ETSPVwHhprdtka
qkLcyWl/M3ijyuAkdYbl
=Q+Du
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-15 Thread Terry Reedy


On 7/15/2013 12:17 AM, Nick Coghlan wrote:


You'd be surprised how many  non-core devs react with astonishment
when I suggest that not documenting something isn't enough to avoid
having users consider it a supported public API - they usually get it
after I point out how far you can usually get just by using dir() and
help() to play with a new library at the interactive prompt instead of
looking at out-of-band docs. I figure including the second part will
help prevent some But why? reactions in the future.


I agree with the goal of preventing such reactions, but I suggest 
slightly different means to do so.


I just looked and idlelib.__init__.py consists of
# Dummy file to make this a package.

help(idlelib) prints the comment (which surprised me, but it seems to be 
smart enough to look for a 'doc comment' when it does not find a doc 
string). I now plan to change the comment to a docstring so 
'idlelib.__doc__' also works and change the text to something like


A package of private modules that implement the IDLE integrated shell 
and editor application. See the manual for the public entry points. 
(or maybe list them briefly here in the doc string).


This should properly inform people who introspect.

Also, the manual needs a small, indexed, section on idlelib that says 
much the same thing along with documenting what *is* public (which is 
not completely clear to me at the moment).



I think Steven has a reasonable point about being clearer that an
explicit leading underscore is the preferred solution for any new
private modules, though, so here's an updated proposal:


The public/private distinction is not always as clean-cut as seems to be 
assumed in this discussion. Test and idlelib both have public entry 
points to an application based on private files. ('Running the Python 
test suite' is a batch application.)


So I do not think either should be _ prefixed. For both, and anything 
similar, a proper docstring and manual entry explaining the nuances 
should be completely acceptable as a substitute for a blunt, 
all-or-nothing flag.


I also think that marking something private, either way, or partially 
private with words, should extend to its contents. In particular, the 
100s of implementation files in test and idlelib need not be _ marked as 
as they would be covered once test and idlelib are properly documented. 
The special _ mark makes sense when it is the exception, but not in a 
context where it would be the overwhelming norm, because everything but 
one or two files is private.



=
Private interfaces

Unless explicitly documented otherwise, a leading underscore on any
name indicates that it is an internal implementation detail and any
backwards compatibility guarantees do not apply. It is strongly
encouraged that private APIs (whether modules, classes, functions,
attributes or other names) be clearly marked in this way, as Python
users may rely on introspection to identify available functionality
and may be misled into believing an API without a leading underscore
is in fact a public API with the standard backwards compatibility
guarantees.

While the explicit use of a leading underscore in the names of private
modules is preferred, all test modules and all modules that are not
explicitly covered in the documentation are also considered private
interfaces, even when their names do not start with a leading
underscore and even if they include a module level documentation
string. This includes submodules of packages that are documented as if
they were a single module.


Since I am planning to explicitly cover idlelib in the doc, to list the 
public interface, just as is done for test, I would like it explicitly 
mentioned along with test.


I do not think absence of documentation is a good signal. I would rather 
say that any private or partially private package that does not have a _ 
name *must* have a docstring saying that it is all or mostly  private. 
(Or be a subpackage or module of something so marked.) And presuming 
that it is associated with something that is documented, then the doc 
should state that the implementation files are private.


In other words, I suggest that modules that are not explicitly covered 
in the documentation ... [whose] names do not start with a leading 
underscore should be empty. And I think even if they include a module 
level documentation string. is backwards. Such files *should* have a 
docstring that serves as a substitute for a _ prefix.



Similarly, the specific modules and external APIs imported by a module
are always considered an implementation detail. Other modules should
not rely on indirect access to such imported interfaces unless they
are an explicitly documented part of the containing module's API (such
as ``os.path`` or a package ``__init__`` module that exposes
functionality from submodules).
=


--
Terry Jan Reedy

___
Python-Dev mailing list

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-15 Thread Barry Warsaw

On Jul 14, 2013, at 06:11 PM, Nick Coghlan wrote:

Private interfaces

PEP 8 does say:

_single_leading_underscore: weak internal use indicator. E.g. from M
import * does not import objects whose name starts with an underscore.

I'm in favor of making this a stronger recommendation, but I have a small
semantic quibble.  Instead of private interface it should be non-public
interface.  The two aren't quite the same thing; for example, often
single-leading underscores are used for methods that subclasses are supposed
to override, e.g. akin to protected in C++.

Besides, Python doesn't really have any strong notion of privateness, so
saying non-public means just because you can, doesn't mean you should.

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-15 Thread Antoine Pitrou

On Mon, 15 Jul 2013 15:51:31 -0400
Barry Warsaw ba...@python.org wrote:
 On Jul 14, 2013, at 06:11 PM, Nick Coghlan wrote:
 
 Private interfaces
 
 PEP 8 does say:
 
 _single_leading_underscore: weak internal use indicator. E.g. from M
 import * does not import objects whose name starts with an underscore.
 
 I'm in favor of making this a stronger recommendation, but I have a small
 semantic quibble.  Instead of private interface it should be non-public
 interface.  The two aren't quite the same thing; for example, often
 single-leading underscores are used for methods that subclasses are supposed
 to override, e.g. akin to protected in C++.

C++ doesn't have a monopoly over the definition of private.

 Besides, Python doesn't really have any strong notion of privateness, so
 saying non-public means just because you can, doesn't mean you should.

If it doesn't have any notion of privateness, then it can't have a
notion of publicness either.
If you really want another word (I am personally fine with private),
internal it should be.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On 16 Jul 2013 06:13, Barry Warsaw ba...@python.org wrote:

 On Jul 15, 2013, at 09:56 PM, Antoine Pitrou wrote:

 If you really want another word (I am personally fine with private),
 internal it should be.

 I would be fine with internal, since that's how PEP 8 already classifies
 such names. :)

Yeah, internal is a better word for this.

Cheers,
Nick.


 -Barry
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 http://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-15 Thread Barry Warsaw

Working from what I think is the latest version.

In general, i'd rather be prescriptive of future conventions than descriptive
of current conventions.  It's okay to exempt existing code, and as a general
rule we've never been fond of rewriting existing code to update it to new
standards or APIs.  We don't need to do so here either.

On Jul 15, 2013, at 05:48 PM, Nick Coghlan wrote:

Private interfaces

Internal or Non-public

Unless explicitly documented otherwise, a leading underscore on any
name indicates that it is an internal implementation detail and any
backwards compatibility guarantees do not apply. It is strongly
encouraged that private APIs (whether modules, classes, functions,
attributes or other names) be clearly marked in this way, as Python
users may rely on introspection to identify available functionality
and may be misled into believing an API without a leading underscore
is in fact a public API with the standard backwards compatibility
guarantees.

How about:

All internal interfaces (modules, classes, functions, attributes or other
names) should be prefixed with a single leading underscore.  Such names are
internal implementation details for which no backward compatibility guarantees
are made, unless otherwise specified.

Existing code and other narrowly accepted exceptions may override this
recommendation, in which case the docstrings and/or documentation for such
code must clearly and explicitly state the internal status of the APIs.

Imported names should always be considered an implementation detail.  Other
modules must not rely on indirect access to such imported names unless they
are an explicitly documented part of the containing module's API, such as
``os.path`` or a package's ``__init__`` module that exposes functionality from
submodules.  Public names exported by a module should be include in the
module's ``__all__`` attribute.

While the explicit use of a leading underscore is the preferred solution,
the names of some private (or partially private) modules (such as ``test``
and ``idlelib``) lack the leading underscore either for historical reasons
or because they expose a public command line interface through the
``-m`` switch. Such modules should include an explicit disclaimer in
their module docstring, indicating that they do not use the leading
underscore convention and noting where the definition of the public
API (if any) can be found (the public API definition may also be part
of the module docstring).

As a general principle, the specific modules and external APIs imported by
a module are always considered an implementation detail. Other modules
should not rely on indirect access to such imported interfaces unless they
are an explicitly documented part of the containing module's API (such
as ``os.path`` or a package ``__init__`` module that exposes functionality
from submodules).

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On Mon, 2013-07-15 at 18:40 -0400, Barry Warsaw wrote:
 Working from what I think is the latest version.
 
 In general, i'd rather be prescriptive of future conventions than descriptive
 of current conventions.  It's okay to exempt existing code, and as a general
 rule we've never been fond of rewriting existing code to update it to new
 standards or APIs.  We don't need to do so here either.

FWIW, I'm very skeptical of a PEP 8 guideline that would try to
proscribe that the non-internal API of a module or class would be
defined solely by a naming convention.

If what's being described here does become a rule, there is reason to
believe that future users who treat this PEP as the word-of-god (and
there are a *lot* of them; I hear from people literally every week who
want to PEP8-ify my code in some limited-value-added way) will be
harmed.  They'll be living in a fantasy world where every
non-underscore-prefixed thing is now a defacto API.  But I have lived in
a world where that has not been the case since 1998, and the chance that
I'll go back and change all my public code to satisfy a questionable
introspection convention is pretty slim.

- C


 
 On Jul 15, 2013, at 05:48 PM, Nick Coghlan wrote:
 
 Private interfaces
 
 Internal or Non-public
 
 Unless explicitly documented otherwise, a leading underscore on any
 name indicates that it is an internal implementation detail and any
 backwards compatibility guarantees do not apply. It is strongly
 encouraged that private APIs (whether modules, classes, functions,
 attributes or other names) be clearly marked in this way, as Python
 users may rely on introspection to identify available functionality
 and may be misled into believing an API without a leading underscore
 is in fact a public API with the standard backwards compatibility
 guarantees.
 
 How about:
 
 All internal interfaces (modules, classes, functions, attributes or other
 names) should be prefixed with a single leading underscore.  Such names are
 internal implementation details for which no backward compatibility guarantees
 are made, unless otherwise specified.
 
 Existing code and other narrowly accepted exceptions may override this
 recommendation, in which case the docstrings and/or documentation for such
 code must clearly and explicitly state the internal status of the APIs.
 
 Imported names should always be considered an implementation detail.  Other
 modules must not rely on indirect access to such imported names unless they
 are an explicitly documented part of the containing module's API, such as
 ``os.path`` or a package's ``__init__`` module that exposes functionality from
 submodules.  Public names exported by a module should be include in the
 module's ``__all__`` attribute.
 
 While the explicit use of a leading underscore is the preferred solution,
 the names of some private (or partially private) modules (such as ``test``
 and ``idlelib``) lack the leading underscore either for historical reasons
 or because they expose a public command line interface through the
 ``-m`` switch. Such modules should include an explicit disclaimer in
 their module docstring, indicating that they do not use the leading
 underscore convention and noting where the definition of the public
 API (if any) can be found (the public API definition may also be part
 of the module docstring).
 
 As a general principle, the specific modules and external APIs imported by
 a module are always considered an implementation detail. Other modules
 should not rely on indirect access to such imported interfaces unless they
 are an explicitly documented part of the containing module's API (such
 as ``os.path`` or a package ``__init__`` module that exposes functionality
 from submodules).
 
 Cheers,
 -Barry
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 http://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe: 
 http://mail.python.org/mailman/options/python-dev/lists%40plope.com


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-15 Thread Steven D'Aprano


On 16/07/13 10:23, Chris McDonough wrote:


If what's being described here does become a rule, there is reason to
believe that future users who treat this PEP as the word-of-god (and
there are a *lot* of them; I hear from people literally every week who
want to PEP8-ify my code in some limited-value-added way) will be
harmed.


I sympathise with your pain, but is that not true of every PEP 8 naming 
convention?



They'll be living in a fantasy world where every
non-underscore-prefixed thing is now a defacto API.


If your code has no obvious, documented convention at all for what's internal 
and what is not, they are no worse off.

If you do have a documented convention for internal implementation details, then you are no worse 
off. I have better things to do than PEP8-ify old, working, stable code is a perfectly 
acceptable answer. I have better things to do than PEP9-ify old, working, stable code, but if 
you want to provide regression tests and a working patch, I'll let you do so might be an even 
better one :-)


--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On 16 July 2013 10:23, Chris McDonough chr...@plope.com wrote:
 On Mon, 2013-07-15 at 18:40 -0400, Barry Warsaw wrote:
 Working from what I think is the latest version.

 In general, i'd rather be prescriptive of future conventions than descriptive
 of current conventions.  It's okay to exempt existing code, and as a general
 rule we've never been fond of rewriting existing code to update it to new
 standards or APIs.  We don't need to do so here either.

 FWIW, I'm very skeptical of a PEP 8 guideline that would try to
 proscribe that the non-internal API of a module or class would be
 defined solely by a naming convention.

 If what's being described here does become a rule, there is reason to
 believe that future users who treat this PEP as the word-of-god (and
 there are a *lot* of them; I hear from people literally every week who
 want to PEP8-ify my code in some limited-value-added way) will be
 harmed.  They'll be living in a fantasy world where every
 non-underscore-prefixed thing is now a defacto API.  But I have lived in
 a world where that has not been the case since 1998, and the chance that
 I'll go back and change all my public code to satisfy a questionable
 introspection convention is pretty slim.

Hence the just say it's an internal API in the docstring escape
clause. That's a pretty low bar to meet for PEP 8 compliance, and one
we need, too (for things like test, idlelib and a bundled pip).

I do need to figure out how to tweak the wording to make it clear that
internal nature of an API applies to all contained APIs as well.

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On Tue, 2013-07-16 at 11:25 +1000, Steven D'Aprano wrote:

 
 If your code has no obvious, documented convention at all for what's internal 
 and what is not, they are no worse off.
 
 If you do have a documented convention for internal implementation details, 
 then you are no worse off. I have better things to do than PEP8-ify old, 
 working, stable code is a perfectly acceptable answer. I have better things 
 to do than PEP9-ify old, working, stable code, but if you want to provide 
 regression tests and a working patch, I'll let you do so might be an even 
 better one :-)

Welp, I guess I'm logically boxed in then.  Thanks for showing me the
errors in my thinking.  Should be no problem to manage the updating of
that 500K lines of public code.

/scarcasm

- C


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On 16 July 2013 12:20, Chris McDonough chr...@plope.com wrote:
 On Tue, 2013-07-16 at 11:25 +1000, Steven D'Aprano wrote:


 If your code has no obvious, documented convention at all for what's 
 internal and what is not, they are no worse off.

 If you do have a documented convention for internal implementation details, 
 then you are no worse off. I have better things to do than PEP8-ify old, 
 working, stable code is a perfectly acceptable answer. I have better 
 things to do than PEP9-ify old, working, stable code, but if you want to 
 provide regression tests and a working patch, I'll let you do so might be 
 an even better one :-)

 Welp, I guess I'm logically boxed in then.  Thanks for showing me the
 errors in my thinking.  Should be no problem to manage the updating of
 that 500K lines of public code.

 /scarcasm

How do get from If this doesn't apply to a module, just add something
like 'This is an internal API' or 'This module includes internal APIs,
consult the documentation for the public API' to the module docstring
to updating 500k lines of public code? The version in Barry's email
that you replied to has that escape clause in it, so the fact it was
missing from my original text doesn't justify this reaction.

Cheers,
Nick.

P.S. Note that, while I'm trying to account for it in this particular
case, we're never going to let the fact that many people misuse PEP 8
by considering it as a holy standard that should be followed by all
Python code everywhere stop us from including updates that are valid
specifically for the standard library.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-15 Thread Barry Warsaw

On Jul 15, 2013, at 08:23 PM, Chris McDonough wrote:

On Mon, 2013-07-15 at 18:40 -0400, Barry Warsaw wrote:
 Working from what I think is the latest version.
 
 In general, i'd rather be prescriptive of future conventions than
 descriptive of current conventions.  It's okay to exempt existing code, and
 as a general rule we've never been fond of rewriting existing code to
 update it to new standards or APIs.  We don't need to do so here either.

FWIW, I'm very skeptical of a PEP 8 guideline that would try to
proscribe that the non-internal API of a module or class would be
defined solely by a naming convention.

If what's being described here does become a rule, there is reason to
believe that future users who treat this PEP as the word-of-god (and
there are a *lot* of them; I hear from people literally every week who
want to PEP8-ify my code in some limited-value-added way) will be
harmed.  They'll be living in a fantasy world where every
non-underscore-prefixed thing is now a defacto API.  But I have lived in
a world where that has not been the case since 1998, and the chance that
I'll go back and change all my public code to satisfy a questionable
introspection convention is pretty slim.

I can sympathize with this.  What I tried to get at, but probably didn't
succeed, is to say if it starts with a single underscore, treat it as
internal but not necessarily the converse, i.o.w. if it *doesn't* start with
an underscore that does not imply that it's public.

-Barry


signature.asc
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On Tue, 2013-07-16 at 12:34 +1000, Nick Coghlan wrote:

 How do get from If this doesn't apply to a module, just add something
 like 'This is an internal API' or 'This module includes internal APIs,
 consult the documentation for the public API' to the module docstring
 to updating 500k lines of public code? The version in Barry's email
 that you replied to has that escape clause in it, so the fact it was
 missing from my original text doesn't justify this reaction.

I may be wrong, but as I see it this is a decision to help (although
only conditionally!) a theoretical IDE user or an agressive user of
dir() at the expense of effectively either 1) making existing
maintainers revisit existing code, or 2 ) making maintainers continually
explain to users that their codebase is not PEP8 compliant.

 P.S. Note that, while I'm trying to account for it in this particular
 case, we're never going to let the fact that many people misuse PEP 8
 by considering it as a holy standard that should be followed by all
 Python code everywhere stop us from including updates that are valid
 specifically for the standard library.

Fair enough, technically.  But historical reality is that a lot of
people just haven't read much past the title: Style Guide for Python
Code.  Or at least they appear to have no concept that it's about code
*in the stdlib only*.  People take it way, way too seriously for
non-stdlib code.  But they do, and it takes time to manage that.

Given that misunderstanding, is there a way to divorce stdlib-centric
guidelines like the one being discussed now from PEP8-ness?  I don't
think any amount of marketing effort or reasoned explanation is going to
separate PEP8 from correct code in people's minds at this point.

- C

ps.. the real irritant behind my minirant is this:

OSS developers have spent many months jumping through bw incompat hoops
in Python over the last few years, and it has taken time away from doing
things that provide value.  The less I can do of that, the better, and
Python gets more value too.  That said, I realize that I'm in the
minority because I happen to have a metric ton of public code out there.
But it'd be nice if that was encouraged rather than effectively punished
on the hunch that it might provide some benefit for a theoretical new
user.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On 16 July 2013 13:02, Chris McDonough chr...@plope.com wrote:
 OSS developers have spent many months jumping through bw incompat hoops
 in Python over the last few years, and it has taken time away from doing
 things that provide value.  The less I can do of that, the better, and
 Python gets more value too.  That said, I realize that I'm in the
 minority because I happen to have a metric ton of public code out there.
 But it'd be nice if that was encouraged rather than effectively punished
 on the hunch that it might provide some benefit for a theoretical new
 user.

You, Armin and everyone else that works on the bytes/text boundary are
indeed the hardest hit by the Python 3 transition, and I appreciate
the hard work you have all done to help make that transition as
successful as it has been so far.

However, the fact that people abuse PEP 8 by treating it as all
Python code in the world should follow these rules cannot, and will
not, stop us from continuing to use it to set appropriate guidelines
*for the standard library*.

I'll look into adding some stronger wording at the top making it clear
that while PEP 8 is a useful starting point and a good default if a
project doesn't have a defined style guide of it's own, it is *not*
the be-all-and-end-all for Python style guides. Treating it as such as
an abuse of the PEP, pure and simple.

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On Tue, 2013-07-16 at 13:11 +1000, Nick Coghlan wrote:
On 16 July 2013 13:02, Chris McDonough chr...@plope.com wrote:
OSS developers have spent many months jumping through bw incompat hoops
in Python over the last few years, and it has taken time away from doing
things that provide value. The less I can do of that, the better, and
Python gets more value too. That said, I realize that I'm in the
minority because I happen to have a metric ton of public code out there.
But it'd be nice if that was encouraged rather than effectively punished
on the hunch that it might provide some benefit for a theoretical new
user.

You, Armin and everyone else that works on the bytes/text boundary are
indeed the hardest hit by the Python 3 transition, and I appreciate
the hard work you have all done to help make that transition as
successful as it has been so far.

However, the fact that people abuse PEP 8 by treating it as all
Python code in the world should follow these rules cannot, and will
not, stop us from continuing to use it to set appropriate guidelines
*for the standard library*.

I'll look into adding some stronger wording at the top making it clear
that while PEP 8 is a useful starting point and a good default if a
project doesn't have a defined style guide of it's own, it is *not*
the be-all-and-end-all for Python style guides. Treating it as such as
an abuse of the PEP, pure and simple.

I understand that. Unfortunately the remainder of the world does not.
The same IDEs that would be helped via this proposed change have PEP8
modes turned on *by default*!
http://blog.jetbrains.com/pycharm/2013/02/long-awaited-pep-8-checks-on-the-fly-improved-doctest-support-and-more-in-pycharm-2-7/

It seems like an unwise step to continue stuffing things into the PEP8
brand bag as a result if that stuff is only meant to apply to the
stdlib.

- C

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On 16 July 2013 13:16, Chris McDonough chr...@plope.com wrote:
I understand that. Unfortunately the remainder of the world does not.
The same IDEs that would be helped via this proposed change have PEP8
modes turned on *by default*!
http://blog.jetbrains.com/pycharm/2013/02/long-awaited-pep-8-checks-on-the-fly-improved-doctest-support-and-more-in-pycharm-2-7/

It seems like an unwise step to continue stuffing things into the PEP8
brand bag as a result if that stuff is only meant to apply to the
stdlib.

Clarifying what constitutes an internal interface in a way that
doesn't require renaming anything is a necessary prerequisite for
bundling or bootstrapping the pip CLI in Python 3.4 (as pip exposes
its internal implemetnation API as import pip rather than import
_pip and renaming it would lead to a lot of pointless code churn).
Without that concern, the topic never would have come up.

Yes, this new section may give the PEP 8 prescriptivists more
ammunition to complain about existing projects with different coding
standards not complying with the latest standard library guidelines.
We're not unfamiliar with the complaint, given the number of standard
library modules (some dating from the 20th century) that don't comply
with the current incarnation of PEP 8 circa 2013.

However, PEP 8 is already quite explicit about what we think of that
kind of behaviour:
http://www.python.org/dev/peps/pep-0008/#a-foolish-consistency-is-the-hobgoblin-of-little-minds

PEP 8 is a tool. Like many of the other tools we provide as part of
Python, it can be abused. However, the likelihood of such abuse is
*not* an excuse to avoid using the tool appropriately, it's a reason
to continue the thankless task of trying to educate people on what's
appropriate and what isn't (and making wholesale changes to working
code just to comply with a style guide generally isn't appropriate).

Cheers,
Nick.

--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

Currently, the naming section of PEP 8 doesn't say very much about what a
leading underscore *means* in the Python standard library.

I would like to add a new Private interfaces subsection under Naming
Conventions to say the following:

=
Private interfaces

Unless explicitly documented otherwise, a leading underscore on any name
indicates that it is an internal implementation detail and backwards
compatibility guarantees do not apply. It is strongly encouraged that
private APIs (whether modules, classes, functions, attributes or other
names) be clearly marked in this way, as many Python users rely on
introspection to identify available functionality and may be mislead into
believing an API without the leading underscore is in fact a public API
with the standard backwards compatibility guarantees.

All test modules are also considered private interfaces.

Even though they typically lack the leading underscore, modules imported by
another module are also considered an implementation detail. Other modules
*should* not rely on indirect access to such modules unless they are an
explicitly documented part of the API (such as ``os.path``).
=

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On 14 July 2013 18:11, Nick Coghlan ncogh...@gmail.com wrote:

 Currently, the naming section of PEP 8 doesn't say very much about what a
 leading underscore *means* in the Python standard library.

 I would like to add a new Private interfaces subsection under Naming
 Conventions to say the following:

 =
 Private interfaces

 Unless explicitly documented otherwise, a leading underscore on any name
 indicates that it is an internal implementation detail and backwards
 compatibility guarantees do not apply. It is strongly encouraged that
 private APIs (whether modules, classes, functions, attributes or other
 names) be clearly marked in this way, as many Python users rely on
 introspection to identify available functionality and may be mislead into
 believing an API without the leading underscore is in fact a public API
 with the standard backwards compatibility guarantees.

 All test modules are also considered private interfaces.

 Even though they typically lack the leading underscore, modules imported
 by another module are also considered an implementation detail. Other
 modules *should* not rely on indirect access to such modules unless they
 are an explicitly documented part of the API (such as ``os.path``).
 =


Slight adjustment to the proposed wording to ensure completely undocumented
modules are also considered private:

=
Private interfaces

Unless explicitly documented otherwise, a leading underscore on any name
indicates that it is an internal implementation detail and backwards
compatibility guarantees do not apply. It is strongly encouraged that
private APIs (whether modules, classes, functions, attributes or other
names) be clearly marked in this way, as Python users may rely on
introspection to identify available functionality and may be misled into
believing an API without a leading underscore is in fact a public API with
the standard backwards compatibility guarantees.

Even when their names do not start with a leading underscore, all test
modules and all modules that are not covered in the documentation are also
considered private interfaces.

Similarly, the specific modules and external APIs imported by a module are
always considered an implementation detail. Other modules should not rely
on indirect access to such imported interfaces unless they are an
explicitly documented part of the containing module's API (such as
``os.path`` or a package ``__init__`` module exposing functionality from
submodules).
=



-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-14 Thread Gregory P. Smith

+1  This is already how we've been behaving for years.


On Sun, Jul 14, 2013 at 4:09 AM, Nick Coghlan ncogh...@gmail.com wrote:

 On 14 July 2013 18:11, Nick Coghlan ncogh...@gmail.com wrote:

 Currently, the naming section of PEP 8 doesn't say very much about what a
 leading underscore *means* in the Python standard library.

 I would like to add a new Private interfaces subsection under Naming
 Conventions to say the following:


 Slight adjustment to the proposed wording to ensure completely
 undocumented modules are also considered private:

 =
 Private interfaces

 Unless explicitly documented otherwise, a leading underscore on any name
 indicates that it is an internal implementation detail and backwards
 compatibility guarantees do not apply. It is strongly encouraged that
 private APIs (whether modules, classes, functions, attributes or other
 names) be clearly marked in this way,


snip


 as Python users may rely on introspection to identify available
 functionality and may be misled into believing an API without a leading
 underscore is in fact a public API with the standard backwards
 compatibility guarantees.


While true, I'm not sure the last part of the sentence is necessary. Once
we've established that a leading _ indicates something is private there
isn't much point in explaining the various ways people might find them.
 I'm happy regardless of this bit being there.


 Even when their names do not start with a leading underscore, all test
 modules and all modules that are not covered in the documentation are also
 considered private interfaces.

 Similarly, the specific modules and external APIs imported by a module are
 always considered an implementation detail. Other modules should not rely
 on indirect access to such imported interfaces unless they are an
 explicitly documented part of the containing module's API (such as
 ``os.path`` or a package ``__init__`` module exposing functionality from
 submodules).
 =

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-14 Thread Cameron Simpson

On 15Jul2013 09:48, Steven D'Aprano st...@pearwood.info wrote:
| On 14/07/13 21:09, Nick Coghlan wrote:
| 
| Slight adjustment to the proposed wording to ensure completely undocumented
| modules are also considered private:
| 
| -1 on this adjustment. If somebody cannot be bothered writing a one-line doc 
string:
| 
| This module is private, don't touch.
| 
| then they certainly shouldn't be allowed to use up a public name for a 
private module. I don't think we should be encouraging more private, 
undocumented modules. (Documentation is valuable even for private modules.)
| 
| I'd go further, and say that no more private modules should be accepted for 
the std lib unless they have a leading underscore. I suppose for backwards 
compatibility reasons, we probably can't go through the std lib and rename 
private modules to make it clear they are private, but we don't have to accept 
new ones without the underscore.

I disagree.

A private module is a perfectly sane way to implement the internals
of something, especially if it is subject to implementation change
in the future.

Clarification: is Nick classifying a module with docstrings by no
content in the modules section of the Python doco as undocumented?
That is what I would presume; I'd expect the code to be littered
with docstrings anyway, but the module as a whole is not presented
in the documentation and so should be private and not relied upon.

Cheers,
-- 
Cameron Simpson c...@zip.com.au

I sometimes wish that people would put a little more emphasis upon the
observance of the law than they do upon its enforcement. - Calvin Coolidge
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-14 Thread Brett Cannon

On Sun, Jul 14, 2013 at 8:01 PM, Cameron Simpson c...@zip.com.au wrote:

 On 15Jul2013 09:48, Steven D'Aprano st...@pearwood.info wrote:
 | On 14/07/13 21:09, Nick Coghlan wrote:
 |
 | Slight adjustment to the proposed wording to ensure completely
 undocumented
 | modules are also considered private:
 |
 | -1 on this adjustment. If somebody cannot be bothered writing a one-line
 doc string:
 |
 | This module is private, don't touch.
 |
 | then they certainly shouldn't be allowed to use up a public name for a
 private module. I don't think we should be encouraging more private,
 undocumented modules. (Documentation is valuable even for private modules.)
 |
 | I'd go further, and say that no more private modules should be accepted
 for the std lib unless they have a leading underscore. I suppose for
 backwards compatibility reasons, we probably can't go through the std lib
 and rename private modules to make it clear they are private, but we don't
 have to accept new ones without the underscore.

 I disagree.

 A private module is a perfectly sane way to implement the internals
 of something, especially if it is subject to implementation change
 in the future.

 Clarification: is Nick classifying a module with docstrings by no
 content in the modules section of the Python doco as undocumented?


Yes. This has nothing to do with docstrings, just the official
documentation at docs.python.org.


 That is what I would presume; I'd expect the code to be littered
 with docstrings anyway, but the module as a whole is not presented
 in the documentation and so should be private and not relied upon.


Exactly.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On 15 July 2013 09:48, Steven D'Aprano st...@pearwood.info wrote:
 On 14/07/13 21:09, Nick Coghlan wrote:

 Slight adjustment to the proposed wording to ensure completely
 undocumented
 modules are also considered private:


 -1 on this adjustment. If somebody cannot be bothered writing a one-line doc
 string:

 This module is private, don't touch.

 then they certainly shouldn't be allowed to use up a public name for a
 private module. I don't think we should be encouraging more private,
 undocumented modules. (Documentation is valuable even for private modules.)

For context, this arose when I checked PEP 8 after the point was
raised on distutils-sig that pip uses a public name for its
implementation module (also pip, same as the CLI), but officially
exposes no stable public programmatic API.

At the moment, according to what little PEP 8 has to say about public
vs private interfaces, that means a bundled pip would represent a new
public API, despite the fact that the only documented interface for
pip is the CLI, not the Python module API.

Since we've been burned by people assuming private APIs are public in
the past, I figured it made sense to get the actual standards we
follow in practice documented properly, *before* anyone further gets
the wrong idea from import pip; help(pip).

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-14 Thread Terry Reedy


On 7/14/2013 7:09 AM, Nick Coghlan wrote:


Slight adjustment to the proposed wording to ensure completely
undocumented modules are also considered private:

=
Private interfaces

Unless explicitly documented otherwise, a leading underscore on any name
indicates that it is an internal implementation detail and backwards
compatibility guarantees do not apply. It is strongly encouraged that
private APIs (whether modules, classes, functions, attributes or other
names) be clearly marked in this way, as Python users may rely on
introspection to identify available functionality and may be misled into
believing an API without a leading underscore is in fact a public API
with the standard backwards compatibility guarantees.

Even when their names do not start with a leading underscore, all test
modules and all modules that are not covered in the documentation are
also considered private interfaces.


I was going to suggest adding 'and most idlelib ' between 'test' and 
'modules', but the broader addition covers idlelib, which is not 
mentioned yet in the docs, even in the (unindexed) Idle chapter. When it 
is, I will try to remember to make explicit which names and interfaces 
are public and that the rest are private.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-14 Thread Steven D'Aprano

On Mon, Jul 15, 2013 at 10:30:02AM +1000, Nick Coghlan wrote:
 On 15 July 2013 09:48, Steven D'Aprano st...@pearwood.info wrote:
  On 14/07/13 21:09, Nick Coghlan wrote:
 
  Slight adjustment to the proposed wording to ensure completely
  undocumented
  modules are also considered private:
 
 
  -1 on this adjustment. If somebody cannot be bothered writing a one-line doc
  string:
 
  This module is private, don't touch.
 
  then they certainly shouldn't be allowed to use up a public name for a
  private module. I don't think we should be encouraging more private,
  undocumented modules. (Documentation is valuable even for private modules.)
 
 For context, this arose when I checked PEP 8 after the point was
 raised on distutils-sig that pip uses a public name for its
 implementation module (also pip, same as the CLI), but officially
 exposes no stable public programmatic API.

I don't think we should bless this as official policy, certainly not for 
the standard library. We can't tell external modules what to do, and 
possibly there's nothing we can do about existing undocumented private 
modules, short of renaming them, but I think that at least for new 
additions to the std lib module names should follow the same 
single-underscore convention as functions, classes etc. A single leading 
underscore introduces the private namespace, everything else is public.

I've seen too much undocumented public code to ever assume that lack of 
documentation implies it is private.


 At the moment, according to what little PEP 8 has to say about public
 vs private interfaces, that means a bundled pip would represent a new
 public API, despite the fact that the only documented interface for
 pip is the CLI, not the Python module API.
 
 Since we've been burned by people assuming private APIs are public in
 the past, I figured it made sense to get the actual standards we
 follow in practice documented properly, *before* anyone further gets
 the wrong idea from import pip; help(pip).

I can tell you that people will do that regardless of what PEP 8 has to 
say about not documented == private. Especially as worded. I'm sure I 
won't be the only person to reason that if a module has a docstring, it 
is therefore documented.

It takes one character to explicitly document a module as private. 
That's much better than expecting people to treat their failure to 
find documentation as an implicit warning.



-- 
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-14 Thread Steven D'Aprano

On Mon, Jul 15, 2013 at 10:01:17AM +1000, Cameron Simpson wrote:
 On 15Jul2013 09:48, Steven D'Aprano st...@pearwood.info wrote:

 | I'd go further, and say that no more private modules should be 
 | accepted for the std lib unless they have a leading underscore. I 
 | suppose for backwards compatibility reasons, we probably can't go 
 | through the std lib and rename private modules to make it clear they 
 | are private, but we don't have to accept new ones without the 
 | underscore.
 
 I disagree.
 
 A private module is a perfectly sane way to implement the internals
 of something, especially if it is subject to implementation change
 in the future.

Of course private modules are sane. I never suggested no new private 
modules at all. But putting them in the same namespace as public 
modules is not, just to save a leading underscore in the file name.

You don't even have to use the underscore in your own code:

import _stuff as stuff

is allowed, and doesn't make _stuff.py public since imported modules are 
considered implementation details by default.


-- 
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

On 15 Jul 2013 13:43, Steven D'Aprano st...@pearwood.info wrote:

 On Mon, Jul 15, 2013 at 10:30:02AM +1000, Nick Coghlan wrote:
  On 15 July 2013 09:48, Steven D'Aprano st...@pearwood.info wrote:
   On 14/07/13 21:09, Nick Coghlan wrote:
  
   Slight adjustment to the proposed wording to ensure completely
   undocumented
   modules are also considered private:
  
  
   -1 on this adjustment. If somebody cannot be bothered writing a
one-line doc
   string:
  
   This module is private, don't touch.
  
   then they certainly shouldn't be allowed to use up a public name for a
   private module. I don't think we should be encouraging more private,
   undocumented modules. (Documentation is valuable even for private
modules.)
 
  For context, this arose when I checked PEP 8 after the point was
  raised on distutils-sig that pip uses a public name for its
  implementation module (also pip, same as the CLI), but officially
  exposes no stable public programmatic API.

 I don't think we should bless this as official policy, certainly not for
 the standard library. We can't tell external modules what to do, and
 possibly there's nothing we can do about existing undocumented private
 modules, short of renaming them, but I think that at least for new
 additions to the std lib module names should follow the same
 single-underscore convention as functions, classes etc. A single leading
 underscore introduces the private namespace, everything else is public.

 I've seen too much undocumented public code to ever assume that lack of
 documentation implies it is private.


  At the moment, according to what little PEP 8 has to say about public
  vs private interfaces, that means a bundled pip would represent a new
  public API, despite the fact that the only documented interface for
  pip is the CLI, not the Python module API.
 
  Since we've been burned by people assuming private APIs are public in
  the past, I figured it made sense to get the actual standards we
  follow in practice documented properly, *before* anyone further gets
  the wrong idea from import pip; help(pip).

 I can tell you that people will do that regardless of what PEP 8 has to
 say about not documented == private. Especially as worded. I'm sure I
 won't be the only person to reason that if a module has a docstring, it
 is therefore documented.

 It takes one character to explicitly document a module as private.
 That's much better than expecting people to treat their failure to
 find documentation as an implicit warning.

We're considering bundling the pip CLI for the convenience of educators and
beginners, not adding their implementation module to the standard library.
That means we need a bright shining line for what constitutes in the
standard library. Yes, it's better if people properly use a leading
underscore, but the fact is many don't (including idlelib, as Terry noted)
and it becomes a disruptive change to add it later if people don't start
with it. The pip and IDLE developers have better things to be doing than
changing the names of undocumented APIs just because the application is
distributed along with CPython.

Cheers,
Nick.




 --
 Steven
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 http://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores