Re: [Python-ideas] PEP 420: implicit namespace sub-package

[Barry Warsaw]

Gallian Colombeau wrote on 8/27/18 06:49:

As I understand, for a package to allow being extended in this way, it 
must be a namespace package and not contain a marker file. As a matter 
of fact, no sub-package until the top level package can have a marker file:


No, that's not true.

However, what is not discussed is "implicit namespace sub-package". In 


There really is no such thing.  Packages are either PEP 420 style 
namespace packages, or regular packages.  The latter contain __init__.py 
files.


The language reference goes into quite a bit of detail on the matter.

https://docs.python.org/3/reference/import.html#packages

Python 3.6 (I guess since the first implementation), if you have this 
layout:

Lib/test/namespace_pkgs
     project1
     parent # Regular package
     __init__.py
     child # Namespace package
     one.py

you get "parent" as a regular package and "parent.child" as a namespace 
package and it works (although now, every package data directory became 
namespace packages and are importable, which may or may not be 
desirable). The point is, does that add any value? 


Personally, I don't think so.  You can do it, but it's not the intended 
purpose, so you're on your own.


I wasn't able to find 
any discussion about this and, as far as I can see, there is actually no 
use case for this as there is no possible way to contribute to the 
"parent.child" namespace. Is that an intended behavior of PEP 420?


There can be use cases for subpackage namespace packages, although they 
are definitely more rare than top-level namespace packages.  One 
possibility would be a plugin system, say for application 'foo', where 
they reserve a subpackage for separate-distribution plugins,  E.g. 
foo.plugins.ext where foo/plugins/ext has no __init__.py file.


Wouldn't it be more appropriate to enforce a sub-package to be a regular 
package if the parent package is a regular package?


As Brett says, it's probably way too late to change this.

-Barry

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add new `Symbol` type

[Barry Warsaw]

Guido van Rossum wrote on 7/6/18 08:31:

Thanks for an interesting discussion. I would also urge people to limit the
use of such sentinels for cases where it is *really* important to
distinguish between, say, f(None) and f(). In most cases using def
f(arg=None) is fine, and often it is even a virtue that passing None or
omitting an argument has exactly the same meaning. (I do know there a cases
where this doesn't apply -- I just think they ought to be fairly unusual.)


One of the most common places I use a non-None sentinel is when None is 
a valid value in a dictionary:


_missing = object()

if mydict.get('foo', _missing) is _missing:
 # it ain't there

I generally don't feel like a more complicated repr is valuable here, so 
I haven't really wanted a built-in sentinel in a long time.  My search 
fu is weak today, but I'm pretty sure I suggested such a thing (and was 
rightly persuaded against it) many years ago.


what-goes-around-comes-around-ly y'rs,
-Barry


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] __dir__ in which folder is this py file

[Barry Warsaw]

On May 15, 2018, at 14:03, Rob Speer  wrote:

> Consider a mini-Web-server written in Python (there are, of course, lots of 
> these) that needs to serve static files. Users of the Web server will expect 
> to be able to place these static files somewhere relative to the directory 
> their code is in, because the files are version-controlled along with the 
> code. If you make developers configure an absolute path, they'll probably use 
> __file__ anyway to get that path, so that it works on more systems than their 
> own without an installer or a layer of configuration management.

You don’t need an absolute path, since you don’t pass file system paths to 
importlib.resources, and even if you relative import a module, you can pass 
that module to the APIs and it will still work, since the loaders know where 
they got the modules from.

> If I understand the importlib.resources documentation, it won't give you a 
> way of accessing your static files directory unless you place an 
> '__init__.py' file in each subdirectory, and convert conventional locations 
> such as "assets/css/main.css" into path(mypackage.assets.css, 'main.css’).

That is correct.  Note that we’re not necessarily saying that we won’t add 
hierarchical path support to the `resource` attributes of the various APIs, but 
they do complicate the semantics and implementation.  It’s also easier to add 
features if the use cases warrant, than remove features that are YAGNI.

> That's already a bit awkward. But do you even want __init__.py to be in your 
> static directory? Even if you tell the mini-server to ignore __init__.py, 
> when you upgrade to a production-ready server like Nginx and point it at the 
> same directory, it won't know anything about this and it'll serve your 
> __init__.py files as static files, leaking details of your system. So you 
> probably wouldn't do this.

Are you saying that servers like Nginx or whatever your mini-server uses don’t 
have a way to blanket ignore files?  That would surprise me, and it seems like 
a lurking security vulnerability regardless of importlib.resources or 
__init__.py files.  I would think that you’d want to whitelist file extensions, 
and that `.py` would not be in that list.

Is this a problem you’ve actually encountered or is it theoretical?

> This is one example; there are other examples of non-Python directories that 
> you need to be able to access from Python code, where adding a file named 
> __init__.py to the directory would cause undesired changes in behavior.

Can you provide more examples?

> Again, importlib.resources is a good idea. I will look into using it in the 
> cases where it applies. But the retort of "well, you shouldn't be using 
> __file__" doesn't hold up when sometimes you do need to use __file__, and 
> there's no universal replacement for it.
> 
> (Also, every Python programmer I've met who's faced with the decision would 
> choose "well, we need to use __file__, so don't zip things" over "well, we 
> need to zip things, so don't use __file__". Yes, it's bad that Python 
> programmers even have to make this choice, and then on top of that they make 
> the un-recommended choice, but that's how things are.)

We certainly see a ton of __file__ usage, but I’m not sure whether it’s the 
case because most developers aren’t aware of the implications, don’t know of 
the alternatives, or just use the simplest thing possible.

Using __file__ in your application, personal web service, or private library is 
fine.  The problem is exacerbated when you use __file__ in your publicly 
released libraries, because not only can’t *you* use them in zip files, but 
nothing that depends on your library can use zip files.  Given how popular pex 
is (and hopefully shiv will be), that will cause pain up the Python food chain, 
and it may mean that other people won’t be able to use your library.

It’s certainly a trade-off, but it’s important to keep this in mind.

If hierarchical resource paths are important to you, I invite you to submit an 
issue to our GitLab project:

https://gitlab.com/python-devs/importlib_resources/issues

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] __dir__ in which folder is this py file

[Barry Warsaw]

Yuval Greenfield wrote:


I often need to reference a script's current directory. I end up writing:

import os
SRC_DIR = os.path.dirname(__file__)


The question I have is, why do you want to reference the script's 
current directory?


If the answer is to access other files in that directory, then please 
consider using importlib.resources (for Python 3.7) and 
importlib_resources (for Python 2.7, 3.4-3.6).


__file__ simply isn't safe, and pkg_resources can be a performance 
killer.  The problem of course is that if you're writing an application 
and *any* of your dependencies use either technique, you are going to 
pay for it.  This is exactly why Brett and I wrote importlib.resources. 
We wanted a consistent API, that allows custom loaders to play along, 
and which is about as efficient as possible, uses Python's import 
machinery, and is safe for uses like zipapps.


now-you-don't-have-to-attend-my-pycon-talk-ly y'rs,
-Barry


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why CPython is still behind in performance for some widely used patterns ?

[Barry Warsaw]

Antoine Pitrou wrote:
> Moore's Law doesn't really apply to semiconductors anymore either, and
> transistor size scaling is increasingly looking like it's reaching its
> end.

You forget about the quantum computing AI blockchain in the cloud.
OTOH, I still haven't perfected my clone army yet.

-Barry

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why CPython is still behind in performance for some widely used patterns ?

[Barry Warsaw]

Just to add another perspective, I find many "performance" problems in
the real world can often be attributed to factors other than the raw
speed of the CPython interpreter.  Yes, I'd love it if the interpreter
were faster, but in my experience a lot of other things dominate.  At
least they do provide low hanging fruit to attack first.

This can be anything from poorly written algorithms, a lack of
understanding about the way Python works, use of incorrect or
inefficient data structures, doing network accesses or other
unpredictable work at import time, etc.  The bottom line I think is that
you have to measure what you've got in production, and attack the
hotspots.  For example, I love and can't wait to use Python 3.7's `-X
importtime` flag to measure regressions in CLI start up times due to
unfortunate things appearing in module globals.

But there's something else that's very important to consider, which
rarely comes up in these discussions, and that's the developer's
productivity and programming experience.  One of the things that makes
Python so popular and effective I think, is that it scales well in the
human dimension, meaning that it's a great language for one person, a
small team, and scales all the way up to very large organizations.  I've
become convinced that things like type annotations helps immensely at
those upper human scales; a well annotated code base can help ramp up
developer productivity very quickly, and tools and IDEs are available
that help quite a bit with that.

This is often undervalued, but shouldn't be!  Moore's Law doesn't apply
to humans, and you can't effectively or cost efficiently scale up by
throwing more bodies at a project.  Python is one of the best languages
(and ecosystems!) that make the development experience fun, high
quality, and very efficient.

Cheers,
-Barry


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Official site-packages/test directory

[Barry Warsaw]

Guido van Rossum wrote:
> IIUC another common layout is to have folders named test or tests inside
> each package. This would avoid requiring any changes to the site-packages
> layout.

That's what I do for all my personal code.  Yes, it means the test
directories are shipped with the sdist, but really who cares?  I don't
think I've had a single complaint about it, even with large-ish projects
like Mailman.  I can see you wanting to do something different if your
project has truly gargantuan test suites, but even with 100% coverage
(or nearly so), I think size just isn't usually a big deal.

In another message, Giampaolo describes being able to run tests with -m
psutil.test.  That's a neat idea which I haven't tried.  But I do think
including the tests can be instructive, and I know that on more than one
occasion, I've cracked open a project's test suite to get a better sense
of the semantics and usage of a particular API.

Finally, I'll disagree with pytest's recommendation to not put
__init__.py files in your test directories.  Although I'm not a heavy
pytest user (we use it exclusive at work, but I don't use it much with
my own stuff), having __init__.py files can be useful, especially if you
also have test data you want to access through pkg_resources, or now,
importlib_resources (importlib.resources in Python 3.7).

Cheers,
-Barry


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax to import modules before running command from the command line

[Barry Warsaw]

Steve Barnes wrote:
> Currently invoking `python -c "some;separated;set of commands;"` will, 
> if you need to use any library functions, require one or more import 
> somelib; sections in the execution string. This results in rather 
> complex "one liners".
> 
> On the other hand `python -m somelib` will load somelib and attempt to 
> execute its `__main__()` or give an error if there isn't one.
> 
> What I would like to suggest is a mechanism to pre-load libraries before 
> evaluating the -c option as this would allow the use of code from 
> libraries that don't have a `__main__` function, or those that do but it 
> doesn't do what you want.

It would be really cool if you could somehow write a file with a bunch
of commands in it, and then get Python to execute those commands.  Then
it could still be a one line invocation, but you could do much more
complex things, including import a bunch of modules before executing
some code.  I'm not sure what such a file would look like but here's a
strawman:

```
import os
import sys
import somelib

path = somelib.get_path()
parent = os.path.dirname(path)
print(parent)

sys.exit(0 if os.path.isdir(parent) else 1)
```

Then you could run it like so:

$ python3 myscript.py

That seems like a nice, compact, one line invocation, but cI don't know,
it probably needs some fleshing out.  It's just a crazy idea, and
there's probably not enough time to implement this for Python 3.7.
Maybe for Python 3.8.

time-machine-winking-ly y'rs,
-Barry


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Repurpose `assert' into a general-purpose check

[Barry Warsaw]

Nathan Schneider wrote:

> I think it would be interesting to investigate how assert statements are
> used in the wild. I can think of three kinds of uses:

assert statements are also a form of executable documentation.  It says
something like "I think the internal state of my code must be this way,
otherwise I don't really understand what I wrote".

Now, maybe you could argue that such use should be enabled
unconditionally, but I disagree since usually your understanding of the
state of your code is correct, so the additional checks are unnecessary,
and people *do* use -O and -OO in practice.  And these days it's even
more practical to do so, given the `opt-*` level of pyc tagging:

% python3 -c "import foo"
% python3 -O -c "import foo"
% python3 -OO -c "import foo"
% ls foo/__pycache__/
__init__.cpython-36.opt-1.pyc   __init__.cpython-36.pyc
__init__.cpython-36.opt-2.pyc

I also wonder how this would interact with pytest's um, 'hijacking' of
the assert statement.

Cheers,
-Barry

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Immutable dictionaries

[Barry Warsaw]

David Mertz wrote:
> https://www.python.org/dev/peps/pep-0416/

PEP 351 (also rejected) is related to this.

-Barry

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] [Python-Dev] What's the status of PEP 505: None-aware operators?

[Barry Warsaw]

On Nov 29, 2017, at 12:40, David Mertz  wrote:

> I think some syntax could be possible to only "catch" some exceptions and let 
> others propagate.  Maybe:
> 
>val = name.strip()[4:].upper() except (AttributeError, KeyError): -1
> 
> I don't really like throwing a colon in an expression though.  Perhaps some 
> other word or symbol could work instead.  How does this read:
> 
>val = name.strip()[4:].upper() except -1 in (AttributeError, KeyError)

I don’t know whether I like any of this  but I think a more natural 
spelling would be:

   val = name.strip()[4:].upper() except (AttributeError, KeyError) as -1

which could devolve into:

   val = name.strip()[4:].upper() except KeyError as -1

or:

   val = name.strip()[4:].upper() except KeyError # Implicit `as None`

I would *not* add any spelling for an explicit bare-except equivalent.  You 
would have to write:

   val = name.strip()[4:].upper() except Exception as -1

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A proliferation of (un-)Pythonically programmatic pragmas

[Barry Warsaw]

Chris Barker wrote:

> but whatever -- multiple line pragmas are fine, too -- I'm hoping putting
> this much crap in your code will be rare :-)

For sure.  It's not uncommon for you to need two pragmas, e.g. flake8
and coverage, but it's pretty rare to need those *plus* mypy.  It does
happen though.

And while I agree it's better to fix the problem, there are lots of
reasons why you can't.  Multiple-version support in libraries is one
common reason (e.g. this code path will never be taken in Python 2 and
the alternative will never be taken in Python 3), bugs in the tools
(false positives), etc.

Cheers,
-Barry


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A proliferation of (un-)Pythonically programmatic pragmas

[Barry Warsaw]

Gregory P. Smith wrote:
> fwiw, we're going to need the tool name in any pragma anyways so the
> existing thing that should be common is:
> 
> # tool-name: meaningfultoken
> 
> It seems like the only convention that makes sense to me.

One of the things that bother me about end-line comments is that this is
going to blow up line length limits.  I think this could work if such
pragma comments could apply to the following line, and multiline pragmas
would be acceptable.  Then you could have something like:

# flake8: disable=unused-import
# mypy: alias=pathlib2.Path
# coverage: ignore=when>py2.7

> When I saw your flake8 example of "# noqa: F401" I wanted to rip my eyes
> out. Because it didn't mention the tool name *and* it used a numeric code.
> Tool authors: use descriptive names!  Otherwise it is meaningless to anyone
> reading it.  ex:

Hah.  One of the things I never imagined was that folks would throw
around numeric PEP numbers and just expect everyone to have PEP 0
tattooed to the back of their eyelids.  Quick, let's discuss PEP 3128!

Cheers,
-Barry

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A proliferation of (un-)Pythonically programmatic pragmas

[Barry Warsaw]

Brett Cannon wrote:

> And possibly the easiest way to reach them is on the pyqa-dev mailing list.

What's that?  I can't find it on python.org, Gmane, or the Googles.

-Barry


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] A proliferation of (un-)Pythonically programmatic pragmas

[Barry Warsaw]

I love many of the ancillary tools that help improve the quality of my
Python code, including flake8, coverage, and mypy.  Each of these
usually produces some great diagnostics, but none of them are perfect,
so they also produce false positives that have to be suppressed.

Each of these tools supports "pragmas", i.e. comments you can add to a
line of code to suppress a warning.  E.g. with flake8 I can say:

import readline, rlcompleter# noqa: F401

to suppress the flake8 warnings about unused imports.  These modules
work by side-effect, which of course a static analyzer can't know.

Similarly, when mypy complains about a line it's confused on, I can add
a comment to suppress the useless warning:

try:
from pathlib import Path, PurePath
except ImportError:
from pathlib2 import Path, PurePath  # type: ignore

And when I want to boost coverage, but I know that some lines aren't
covered under some versions of Python, I can do something like:

self.send_error(HTTPStatus.NOT_FOUND)   # pragma: nocover

These are all well and good until I have to *combine* suppressions.
E.g. in the pathlib2 pragma to mypy, I also get a flake8 warning and
I've tried just about every combination of pragma comment I can think
of, but I've not been able to make both tools happy.  I've resorted to
refactoring the code into an entire module for flake8 to ignore and added:

# flake8: noqa

to suppress all warnings in the file.  I actually have hit on a few
places where I need to suppress warnings from all three tools on the
same line, and I basically can't do it.

The specifics aren't as important as the general use case: multiple
tools competing for the same valuable real-estate.

I have no ideas how to improve the situation, and of course any solution
would involve some coordination between all of these tools, but it's
beginning to feel like a losing battle.  Is there a better way?

Cheers,
-Barry

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Barry Warsaw]

Guido van Rossum wrote:

> This is beginning to sound like the most attractive solution. We could
> possibly do away with typing_extensions. Are there precedents of how to
> bundle a module in this way? Or is it going to be another special case like
> pip?

With my downstream consumer hat on, *if* we go with a bundling solution,
please make sure it's easy to unbundle.  Downstream consumers may need
to do this for policy, security, etc. reasons, so making this too
difficult will mean delays in releases, convoluted hacks, policy
violations, or all of the above.

See as reference some of the hoops Debian has to go through to handle
ensurepip.

Cheers,
-Barry


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal to change Python version release cycle

[Barry Warsaw]

Nick Timkovich wrote:

> Alternative history question: if it was just 1.6

Guido's time machine strikes again.  There was both a Python 1.6 and a
1.6.1.

https://www.python.org/download/releases/1.6/
https://www.python.org/download/releases/1.6.1/

The "Contractual Obligation" releases.  (And another Monty Python's
Flying Circus reference.)  These resolved licensing and release issues
related to the BeOpen adventure and GPL-compatibility.

Cheers,
-Barry


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal to change Python version release cycle

[Barry Warsaw]

M.-A. Lemburg wrote:

> The first ever major backwards incompatibility release switch we
> had in Python after the great renaming of the C APIs between
> Python 1.1 and 1.2 (which was only visible to C extensions and
> relatively easy to fix using a compatibility header file),
> was the transition from Python 2.x to Python 3.x.

Fond memories of some of my first contributions to Python:

http://python-history.blogspot.com/2009/03/great-or-grand-renaming.html

Cheers,
-Barry



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] PEP Post-History

[Barry Warsaw]

We’ve made a small change to the PEP process which may affect readers of 
python-list and python-ideas, so I’d like to inform you of it.  This change was 
made to PEP 1 and PEP 12.

PEPs must have a Post-History header which records the dates at which the PEP 
is posted to mailing lists, in order to keep the general Python community in 
the loop as a PEP moves to final status.  Until now, PEPs in development were 
supposed to be posted at least to python-dev and optionally to python-list[1].  
This guideline predated the creation of the python-ideas mailing list.

We’ve now changed this guideline so that Post-History will record the dates at 
which the PEP is posted to python-dev and optionally python-ideas.  python-list 
is dropped from this requirement.

python-dev is always the primary mailing list of record for Python development, 
and PEPs under development should be posted to python-dev as appropriate.  
python-ideas is the list for discussion of more speculative changes to Python, 
and it’s often where more complex PEPs, and even proto-PEPs are first raised 
and their concepts are hashed out.  As such, it makes more sense to change the 
guideline to include python-ideas and/or python-dev.  In the effort to keep the 
forums of record to a manageable number, python-list is dropped.

If you have been watching for new PEPs to be posted to python-list, you are 
invited to follow either python-dev or python-ideas.

Cheers,
-Barry (on behalf of the Python development community)

https://mail.python.org/mailman/listinfo/python-dev
https://mail.python.org/mailman/listinfo/python-ideas

Both python-dev and python-ideas are available via Gmane.

[1] PEPs may have a Discussions-To header which changes the list of forums 
where the PEP is discussed.  In that case, Post-History records the dates that 
the PEP is posted to those forums.  See PEP 1 for details.



signature.asc
Description: Message signed with OpenPGP
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] lazy import via __future__ or compiler analysis

[Barry Warsaw]

Neil Schemenauer wrote:

> Introduce a lazy module import process that modules can opt-in to.
> The opt-in would either be with a __future__ statement or the
> compiler would statically analyze the module and determine if it is
> safe.  E.g. if the module has no module level statements besides
> imports.

and `def` and `class` of course.

There are a few other things that might end up marking a module as
"industrious" (my thesaurus's antonym for "lazy").  There will likely be
assignments of module global such as:

MY_CONST = 'something'

and it may even be a little more complicated:

COLORS = dict(
red=1,
blue=2,
green=3,
)
REVERSE = {value: key for key, value in COLORS.items()}

A naive evaluation of such a module might not notice them as lazy, but I
think they could still be treated as such.

Function and class decorators might also be false positives.  E.g.

@public
def my_public_function():
pass

or even

@mungify
class Munged:
pass

Maybe that's just the cost of doing business, and if they clear the lazy
flag, so be it.  But it feels like doing so will leave quite a bit of
lazy loading opportunity left on the table.  And I'm not sure you can
solve all of those by moving things to a module level __init__().

Cheers,
-Barry


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 550 dumbed down

[Barry Warsaw]

Jim J. Jewett wrote:
> I know I'm not the only one who is confused by at least some of the
> alternative terminology choices.  I suspect I'm not the only one who
> sometimes missed part of the argument because I was distracted
> figuring out what the objects were, and forgot to verify what was
> being done and why.  I also suspect that it could be much simpler to
> follow if the API were designed in the abstract, with the
> implementation left for later.

You're definitely not alone!  I think I get the gist of the proposal,
and its motivation, but I'm definitely confused by the terminology.  As
I stated elsewhere, the word "context" has a well-established meaning in
Python, with context managers, their protocols, and contextlib.  When
talking with another Pythonista three years from now, I don't want to
have to resolve which context they're talking about based on context. ;)

I think you have a point too about designing the abstract behavior and
API first, and then worry about implementation details (in fact, maybe
take implementation discussions out of the PEP for now, and maybe hash
that out in a PR).

I also think you're on to something when you suggest that sys may not be
the best place for these new APIs.  sys is already a mishmash of lots of
random stuff, and the concepts defined in PEP 550 are advanced enough
that many Python developers will never need to worry about them.
Putting them in sys leads to cognitive overload.  I'm not sure I'd put
them in builtins either, but a new module makes a lot of sense to me.
Plus, it means that we can choose more natural names for the APIs since
they'll be namespaced away in a separate module.

Cheers,
-Barry


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP 550: Execution Context

[Barry Warsaw]

Yury Selivanov wrote:

> This is a new PEP to implement Execution Contexts in Python.

It dawns on me that I might be able to use ECs to do a better job of
implementing flufl.i18n's translation contexts.  I think this is another
example of what the PEP's abstract describes as "Context managers like
decimal contexts, numpy.errstate, and warnings.catch_warnings;"

The _ object maintains a stack of the language codes being used, and you
can push a new code onto the stack (typically using `with` so they get
automatically popped when exiting).  The use case for this is
translating say a notification to multiple recipients in the same
request, one who speaks French, one who speaks German, and another that
speaks English.

The problem is that _ is usually a global in a typical application, so
in an async environment, if one request is translating to 'fr', another
might be translating to 'de', or even a deferred context (e.g. because
you want to mark a string but not translate it until some later use).

While I haven't used it in an async environment yet, the current
approach probably doesn't work very well, or at all.  I'd probably start
by recommending a separate _ object in each thread, but that's less
convenient to use in practice.  It seems like it would be better to
either attach an _ object to each EC, or to implement the stack of codes
in the EC and let the global _ access that stack.

It feels a lot like `let` in lisp, but without the implicit addition of
the contextual keys into the local namespace.  E.g. in a PEP 550 world,
you'd have to explicitly retrieve the key/values from the EC rather than
have them magically appear in the local namespace, the former of course
being the Pythonic way to do it.

Cheers,
-Barry

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] dict(default=int)

[Barry Warsaw]

On Mar 08, 2017, at 05:49 PM, Eric V. Smith wrote:

>If we really want to make defaultdict feel more "builtin" (and I don't see
>any reason to do so), I'd suggest adding a factory function:
>
>dict.defaultdict(int)
>
>Similar in spirit to dict.fromkeys(), except of course returning a
>defauldict, not a dict.

Nice.

-Barry


pgpK0qyKJMncc.pgp
Description: OpenPGP digital signature
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Optional parameters without default value

[Barry Warsaw]

On Mar 02, 2017, at 06:37 PM, Brett Cannon wrote:

>So to me, there's actually two things being discussed. Do we need another
>sentinel to handle the "None is valid" case, and do we want syntax to more
>clearly delineate optional arguments?

No, and no (IMHO).

-Barry


pgpULmSfZJDcd.pgp
Description: OpenPGP digital signature
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 8 coding style included in grammar ?

[Barry Warsaw]

On Mar 01, 2017, at 03:04 PM, Mathieu BEAL wrote:

>I was wondering why the PEP coding style (
>https://www.python.org/dev/peps/pep-0008/) is not natively included in python
>grammar ?

Well, the simple answer is that the grammar predates PEP 8 (or any PEP) by
many years.

Cheers,
-Barry


pgpGjysEgHeP4.pgp
Description: OpenPGP digital signature
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Efficient debug logging

[Barry Warsaw]

On Feb 16, 2017, at 03:20 PM, M.-A. Lemburg wrote:

>I know some people will disagree, but IMO using "assert" is the wrong
>approach in such situations - it's meant for development and testing
>only, not as short-cut to avoid having to write a proper error
>handler :-)

I use assertions for "things that can never happen", although sometimes they
do due to an incomplete understanding of the code, the problem, or the
environment.

Cheers,
-Barry


pgpJ1sOAxhaoc.pgp
Description: OpenPGP digital signature
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Define a method or function attribute outside of a class with the dot operator

[Barry Warsaw]

On Feb 14, 2017, at 12:48 PM, Steven D'Aprano wrote:

>On Fri, Feb 10, 2017 at 09:05:49PM -0500, Terry Reedy wrote:
>> Saving about 10 keystrokes is close to trivial.  
>
>The same argument can be made for @decorator syntax.
>
>And, if I understand correctly, the same argument *was* made against
>decorator syntax: that it was trivial, unnecessary and confusing.

Well, not exactly.  Remember that the semantics, and common decorators like
property, existed well before the decorator syntax was added.  We had a lot of
experience writing post-definition "decorators", which taught us that the
behavior was useful but the syntax was painful.  And adding the syntax made a
huge improvement in readability.

Cheers,
-Barry


pgpIpdsRZULwq.pgp
Description: OpenPGP digital signature
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Things that won't change (proposed PEP)

[Barry Warsaw]

On Jan 13, 2017, at 11:20 AM, Greg Ewing wrote:

>Criticisms Frequently Levelled Against Python

Missteps Or Nonfeatures Guido Obviously Ordered, Saddling Everyone

(Yes, okay, I know python's aren't venomous, but never let facts get in the
way of a good, bad, tortured, or mentally mushed Friday-evening backronym.)

Cheers,
-Barry


pgpmYyHMRfJQQ.pgp
Description: OpenPGP digital signature
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 540: Add a new UTF-8 mode

[Barry Warsaw]

On Jan 06, 2017, at 11:08 PM, Steve Dower wrote:

>Passing universal_newlines will use whatever locale.getdefaultencoding()

There is no locale.getdefaultencoding(); I think you mean
locale.getpreferredencoding(False).  (See the "Changed in version 3.3" note in
$17.5.1.1 of the stdlib docs.)

>universal_newlines may become a bad choice if the default encoding no longer
>matches what the environment says, and personally, I wouldn't lose much sleep
>over that.

universal_newlines is also problematic because it's misnamed from the more
common motivation to use it.  Very often people do want to open std* in text
mode (and thus trade in Unicodes), but they rarely equate that to "universal
newlines".  So the option is just more hidden magical side-effect and
cargo-culted lore.  It's certainly *useful* though, and I think we want to be
sure that we don't break existing code that uses it for this purpose.

Cheers,
-Barry


pgp0FauJl68GN.pgp
Description: OpenPGP digital signature
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 540: Add a new UTF-8 mode

[Barry Warsaw]

On Jan 06, 2017, at 07:22 AM, Stephan Houben wrote:

>Because I have the impression that nowadays all Linux distributions are UTF-8
>by default and you have to show some bloody-mindedness to end up with a POSIX
>locale.

It can still happen in some corner cases, even on Debian and Ubuntu where
C.UTF-8 is available and e.g. my desktop defaults to en_US.UTF-8.  For
example, in an sbuild/schroot environment[*], the default locale is C and I've
seen package build failures because of this.  There may be other such "corner
case" environments where this happens too.

Cheers,
-Barry

[*] Where sbuild/schroot is a very common suite of package building tools.


pgp8kv4ZZDw4D.pgp
Description: OpenPGP digital signature
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 540: Add a new UTF-8 mode

[Barry Warsaw]

On Jan 05, 2017, at 05:50 PM, Victor Stinner wrote:

>I guess that all users and most developers are more in the "UNIX mode"
>camp. *If* we want to change the default, I suggest to use the "UNIX
>mode" by default.

FWIW, it seems to be a general and widespread recommendation to always pass
universal_newlines=True to Popen and friends when you only want to deal with
unicode from subprocesses:

If encoding or errors are specified, or universal_newlines is true, the
file objects stdin, stdout and stderr will be opened in text mode using
the encoding and errors specified in the call or the defaults for
io.TextIOWrapper.

Cheers,
-Barry


pgplvA9Q9CyK2.pgp
Description: OpenPGP digital signature
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Null coalescing operator

[Barry Warsaw]

On Oct 28, 2016, at 03:24 PM, Gustavo Carneiro wrote:

>The main drawback of this type of approach is that code checking tools will
>hardly ever support checking expressions inside the string like that.
>Also, you don't get proper syntax highlighting, code completion, etc.
>
>You can do anything you want by writing another programming language that
>is passed as string to a function, but that is not the same thing as having
>a proper syntax, is it?  Just like type annotations with mypy: sure, you
>can add type annotations in comments, but it's not the same...

The bar for adding new language syntax is, and must be, high.  Every new bit
of syntax has a cost, so it has to be worth it.  Guido deemed type annotations
to be worth it and he may do the same for null coalescing operators.  I don't
personally think the need is so great or the use cases so common to incur that
cost, but I'm just one opinion.

The advantage of lower-cost approaches such as adopting the syntax in
attrgetter() is that you piggyback on an existing API.  Then you can use that
as an experiment to see whether you really do solve enough problems in Python
for a syntax change to be worth it.  It's a lot like the ability to create
properties and such before the syntactic sugar of decorators was added.  I
think that feature's pre-syntax popular and utility proved that the cost of
adding syntax was worth it.

Cheers,
-Barry


pgp93j7OeA2dm.pgp
Description: OpenPGP digital signature
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Null coalescing operator

[Barry Warsaw]

On Oct 27, 2016, at 07:37 PM, Nick Badger wrote:

>The problem with doing that is that it's ambiguous. There's no way of
>telling which attribute is allowed to coalesce.

You could of course support exactly the same syntax being proposed as a
language change, e.g.

from operator import attrgetter
r = attrgetter('b?.x?.z')

and then you wouldn't even need the `coalesce` argument.

Cheers,
-Barry


pgp6gsyF_aVVA.pgp
Description: OpenPGP digital signature
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Distribution agnostic Python project packaging

[Barry Warsaw]

On Oct 27, 2016, at 02:50 PM, James Pic wrote:

>Now I'm fully aware of distribution specific packaging solutions like
>dh-virtualenv shared by Spotify but here's my mental problem: I love to
>learn and to hack. I'm always trying now distributions and I rarely run the
>one that's in production in my company and when I'm deploying personal
>projects I like funny distributions like arch, Alpine Linux,  or
>interesting paas solutions such as cloudfoundry, openshift, rancher and
>many others.
>
>So that's the idea I'm trying to share: I'd like to b able to build a file
>with my dependencies and my project in it.

You might want to look at the Snap ecosystem.  It's fairly new, but it is
cross-distro and cross-arch, and in many ways a very cool way to build
self-contained applications where you control all the dependencies.  You don't
have to worry so much about each distribution's peculiarities, and Python gets
first-class support[*].

There are lots of technical and philosophical aspects to Snaps that are
off-topic for this mailing list, so I'll just point you to where you can
explore it on your own:

http://snapcraft.io/

Disclosure: I work for Canonical in my day job, which invented the technology,
but it is in very large part an open source community project.

Cheers,
-Barry

[*] In fact, the nice convenience front-end to building snaps is a Python 3
application.


pgpVdSY4u4DeG.pgp
Description: OpenPGP digital signature
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] real numbers with SI scale factors: next steps

[Barry Warsaw]

On Aug 30, 2016, at 02:16 PM, Guido van Rossum wrote:

>Given that something like this gets proposed from time to time, I
>wonder if it would make sense to actually write up (1) and (2) as a
>PEP that is immediately marked rejected. The PEP should make it clear
>*why* it is rejected. This would be a handy reference doc to have
>around the next time the idea comes up.

There certainly is precedence: e.g. PEPs 404 and 666. :)

Cheers,
-Barry


pgpLLpPQDzqcA.pgp
Description: OpenPGP digital signature
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/