[Python-Dev] Re: PEP 689 – Semi-stable C API tier

[Glenn Linderman]

On 4/29/2022 11:42 AM, Stephen J. Turnbull wrote:

MRAB writes:
  > On 2022-04-29 18:02, Guido van Rossum wrote:
  > > On Fri, Apr 29, 2022 at 10:15 AM Petr Viktorin  > > wrote:
  > >
  > > On 29. 04. 22 16:32, Victor Stinner wrote:
  > >  > Ok, let me start with the serious business: API name.
  > >  >
  > >  > I'm not comfortable with "semi-stable". Python already has a 
"limited
  > >  > API" and a "stable ABI". Just by its name, it's unclear what
  > >  > "semi-stable" means.
  > >  >
  > >  > Honestly, I would be more comfortable with the name: "unstable 
API".
  > >  > It would be clear that the API *can* change often. People who want 
to
  > >  > know exactly the backward compatibility warranties can dig into the
  > >  > API documentation to learn more about it.
  > >  >
  > >  > "Unstable API" is also the name the Guido proposed for
  > > PyCode_New() last year:
  > >  >
  > >  > * Proposal: declare "unstable APIs"
  > >  >
  > > 
https://mail.python.org/archives/list/python-dev@python.org/thread/JM6SQ2YNMDAKXYD5O54QWMVR2X7QOXVL/
  > > 

  > >  > * Making code object APIs unstable
  > >  >
  > > 
https://mail.python.org/archives/list/python-dev@python.org/thread/ZWTBR5ESYR26BUIVMXOKPFRLGGYDJSFC/
  > > 

  > >  >
  > >  > Victor
  > >
  > >
  > > Nick Coghlan argued against that term:
  > >
  > >  > "unstable" is the wrong term. We already have an unstable API
  > > tier: the
  > >  > internal API, which can change even in maintenance releases. The
  > > value of
  > >  > the new tier is that it is "semi stable": stable in maintenance
  > > releases,
  > >  > unstable in feature releases.
  > >
  > > —
  > > 
https://mail.python.org/archives/list/python-dev@python.org/message/CTKKTHUV5R2A2RRN5DM32UQFNC42DDGJ/
  > > 

  > >
  > >
  > > But I also like “unstable” better than “semi-stable”. Splitting the
  > > internals into “private”/“internal” and “unstable” seems reasonable.
  > >
  > >
  > > I think picking "semi-stable" would be giving in to the OCD nerd in all
  > > of us. :-) While perhaps technically less precise, "unstable" is the
  > > catchy name with the right association. (And yes, we should keep it
  > > stable within bugfix releases, but the name doesn't need to reflect that
  > > detail.) The "internal API" isn't an API at all (except for CPython core
  > > developers and contributors). The "unstable API" would definitely be an
  > > *API* for users outside the core.
  > >
  > > So let's please go with "unstable".
  > >
  > I was going to suggest "metastable". Too late? :-)
A
Bikeshedding to a new level! --+


Metabikeshedding...


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CGNNVU7CUB4U6TK33FSYVIGP7OPPG4XJ/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GCS2AC32ODMY44PA6AS6JLLCJBP55BZV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 646 (Variadic Generics): final call for comments

[Glenn Linderman]

On 1/18/2022 2:35 PM, Jim J. Jewett wrote:

The problem is that
 [*s3, *s4] = (a, b, 1, 2, 3)
is ambiguous ...

It wouldn't have to be... but as you say, it needs to be explicit.

*s3 could get all the content, and *s4 a tuple with no members.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CNTEQGZRHLPODTIR74SUHD7OWSEZ4FL3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Suggestion: a little language for type definitions

[Glenn Linderman]

On 1/8/2022 2:05 PM, Skip Montanaro wrote:

> ... make sense of what they’re reading.

Some of us have that problem with type-embellished code now.I'm not 
sure a little language would be such a bad idea. 樂Fortunately, my 
relationship to the working world allows me to simply ignore explicit 
typing. 


Way, way BITD I recall leaning on a crutch to generate complex C type 
declarations. I no longer recall what it was called, but you gave it a 
restricted English description of what you wanted ("function returning 
pointer to function returning void pointer" or something similar) and 
it spit out the necessary line noise.


Skip


Oh yes, I remember that... went Googling and found https://cdecl.org/ 
which is apparently the "modern" online version of that old tool.


I guess typing features might be useful for people that refuse to 
write/read comments, but I mostly ignore the typing stuff too... and 
hope I can continue to.___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6XLUYYGPIPXMSRM5NTW4QVU3SDG6SRX4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 except* formatting

[Glenn Linderman]

On 10/3/2021 10:23 PM, Guido van Rossum wrote:

On Sun, Oct 3, 2021 at 9:20 PM Jonathan Goble  wrote:

Therefore my vote is for requiring `except* E` and keeping `except
*E` as a SyntaxError.


You can't do that with our current lexer+parser.


Seems like a good reason to promote   "except group E"  instead of 
"except * E", as others have suggested.___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RHUNTWTPDJS5M4KPGHCRV4H34BD26BFO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Roundup to GitHub Issues migration

[Glenn Linderman]

On 6/22/2021 12:52 PM, Brett Cannon wrote:



On Mon, Jun 21, 2021 at 4:09 PM Glenn Linderman <mailto:v%2bpyt...@g.nevcal.com>> wrote:


On 6/21/2021 2:31 PM, Christopher Barker wrote:


By contrast, requiring a github account for reporting bugs
also makes
python an unwelcoming place for non-developers in general.
Github is a
developers' social network, "mere" users are much less likely
to want to
be part of it. Many will just silently abandon their bug report.


But you don’t need to be “part of it” in any meaningful way. One
only needs to create an account, which could be quite anonymous,
and even temporary.

And is no harder, and probably easier, than creating an account
on a Python-specific site.

Also: cPython is a large, complex, and mature project. I don't
think many non-developers can even identify a true bug, much less
write a helpful big report. There are many other ways to be
involved in and contribute to the Python community that don't
require a gitHub (or any) account.

I understand the issue here — I feel that way about businesses
that use Facebook for their website. But in that case, I can’t
even read it without a Facebook account. I don’t mind needing an
account to contribute to a conversation.

And while GitHub  has become the dominant player in Open Source
development— it has not (yet?) reached out to control much else.

-CHB


With all due respect to Microsoft, who has contributed
significantly to Python development, and continues to do, some
people don't care for some of Microsoft's policy and actions, and
Microsoft owns GitHub, so your last paragraph is somewhat naive,
at best.

So what is the difference between a GitHub account, and Microsoft
account?


They are entirely different and separate; there's no relation there at 
all.


Good, but given the Skype experience, it is questionable how long they 
will stay that way.


One thing I will remind people is I personally have led the work to 
move this project from:


 1. SourceForge to our own infrastructure
 2. Mercurial to git
 3. Our own infrastructure to GitHub for code management

So if GitHub were to do something we don't like we can always move 
platforms again.


Indeed, but platform churn slows contributions, too.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5ST6GPPTTCXSRE26COVMH4A4KQH2CVHM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Roundup to GitHub Issues migration

[Glenn Linderman]

On 6/21/2021 2:31 PM, Christopher Barker wrote:


By contrast, requiring a github account for reporting bugs also makes
python an unwelcoming place for non-developers in general. Github is a
developers' social network, "mere" users are much less likely to
want to
be part of it. Many will just silently abandon their bug report.


But you don’t need to be “part of it” in any meaningful way. One only 
needs to create an account, which could be quite anonymous, and even 
temporary.


And is no harder, and probably easier, than creating an account on a 
Python-specific site.


Also: cPython is a large, complex, and mature project. I don't think 
many non-developers can even identify a true bug, much less write a 
helpful big report. There are many other ways to be involved in and 
contribute to the Python community that don't require a gitHub (or 
any) account.


I understand the issue here — I feel that way about businesses that 
use Facebook for their website. But in that case, I can’t even read it 
without a Facebook account. I don’t mind needing an account to 
contribute to a conversation.


And while GitHub  has become the dominant player in Open Source 
development— it has not (yet?) reached out to control much else.


-CHB


With all due respect to Microsoft, who has contributed significantly to 
Python development, and continues to do, some people don't care for some 
of Microsoft's policy and actions, and Microsoft owns GitHub, so your 
last paragraph is somewhat naive, at best.


So what is the difference between a GitHub account, and Microsoft 
account?  Skype used to have its own accounts... then they were 
converted to be Microsoft accounts...


Glenn
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AQXZFCI7EQ6JBEH6VKTHUKSCB5WRP7B3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: declare "unstable APIs"

[Glenn Linderman]

On 6/3/2021 3:34 PM, Guido van Rossum wrote:
On Thu, Jun 3, 2021 at 3:17 PM Tim Delaney 
mailto:timothy.c.dela...@gmail.com>> wrote:


Perhaps "living API" analogous to "living document". Much more
positive connotations ...


Perhaps, but that's pretty much coining a new term, which we would 
then have to explain. And since the opposite would be "dead API", what 
is a dead API exactly? And doesn't it follow that all APIs are living? 
If we stick with unstable, we're adopting a term that's in use by at 
least one other language community (Rust), and the slightly negative 
connotation is welcome -- people should think twice before using 
unstable APIs.


I read somewhere that the term "stable" means "dead" in some contexts... 
was it maybe a medical context?  So "living" would be "unstable" too, as 
Tim suggested.


And since people know what a living document is, a living API wouldn't 
be much of a stretch.


On the other hand, "unstable" carries a bit more connotation of "needs 
caution".
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CS23XQX3356Q3S6WI66PISIT3NXEGMGM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: name for new Enum decorator

[Glenn Linderman]

On 6/2/2021 7:59 PM, Ethan Furman wrote:

On 5/27/21 8:24 PM, Ethan Furman wrote:

> So, like the enum.unique decorator that can be used when duplicate 
names should be an error,
> I'm adding a new decorator to verify that a Flag has no missing 
aliased values that can be
> used when the programmer thinks it's appropriate... but I have no 
idea what to call it.

>
> Any nominations?

Thank you everyone for your ideas!  Instead of adding another 
single-purpose decorator, I'm toying with the idea of adding a general 
purpose decorator that accepts instructions.  Something along the 
lines of:


    class EnumChecks(StrEnum):
    """
    various conditions to check an enumeration for
    """
    UNIQUE = "one name per value"
    CONTINUOUS = "no skipped values"
    DECOMPOSABLE_ALIASES = "flag aliases must have all bits named"


    def verify(enumeration, *checks):
    if UNIQUE in checks:
    # ensure no duplicates
    if CONTINUOUS in checks:
    # ensure no missing values
    if DECOMPOSABLE_ALIASES in checks:
    # ensure all multi-flag aliases are composed of named flags

Thoughts?


Seems more forward-looking and extensible rather than a proliferation of 
decorators. I like that.


And the EnumChecks provides a way to make a tradeoff between short names 
(although DECOMPOSABLE_ALIASES isn't particularly short, it could be 
made shorter given the explanations) and the explanations of them in the 
text value.


However, for DECOMPOSABLE_ALIASES, not only is the name not short, the 
explanation isn't particularly clear either, and it doesn't sound like 
it would properly explain the case where you want to have a mask that is 
larger than the number of currently used individual "flags" (which maybe 
shouldn't be called "bits", but "flags").  That explanation confuses the 
terminology between "masks" calling them "flags" and "flags" calling 
them bits.


But maybe my understanding of what you originally meant the term "flag" 
to mean in the Enum context is not clear... in any case, a terminology 
should be used consistently, and the terminology should differentiate 
between at least 3 cases:


1. single bit items  (flag?)
2. multi-bit values that are not a superset of a group of related single 
bit items (group?)  e.g.  PURPLE = RED | BLUE (not including GREEN)
3. multi-bit values that are intended as a mask, to include all related 
single bit items as well as possibly reserving space for future, 
yet-undefined, related single bit items. (mask?)


and maybe also

4. multi-bit fields containing a number, rather than a set of individual 
bits (field?)


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VPAMZXZFKJ5SVGUWSIV6DB5EBHXXLMCV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: name for new Enum decorator

[Glenn Linderman]

On 5/31/2021 1:37 PM, Luciano Ramalho wrote:

On Fri, May 28, 2021 at 7:00 PM Joao S. O. Bueno  wrote:

"check_all_bits_defined" or something along it.

Best suggestion so far. Similar:

"ensure_all_bits_named"

I am all for brainstorming, as we've been doing for a few days. Maybe
we need to brainstorm some more.

I just don't think the goal should be finding a one/two-word name for
the decorator.

I see no need to find something shorter than a short sentence. After
all, defining a new enum is not something we do all the time.

Cheers,

Luciano


In this vein, I suggest these variations

    @name_all_used_flags
    @require_individual_flag_names

Clearly some flags may not be used, and need not be named, but this 
would require all the used flags to be (individually) named, without 
precluding combinations of same.


And while I am guilty of using the word "bits" in prior suggestions, I 
realized today that the name of the Enum variety under discussion is 
"IntFlag" not "IntBits", so it seems more appropriate to use the word 
"flag" or "flags" rather than "bits", even though the implementation of 
a boolean flag is a single bit.


So I withdraw my prior suggestions of  @all_bits_defined  and 
@unique_bits in favor of  @all_flags_defined and @unique_flags.



Digression/Extensions:

I'm also starting to wonder if Flag & IntFlag should be restricted to 
individual flags and combinations, and if another variety of Enum, that 
is very, very similar, should be called IntBitfields.  On the other 
hand, if there are a few flags and a small bitfield or two, that are 
sufficiently related, having them contained in a single Enum might be 
appropriate, and such has certainly been used on pre-existing APIs with 
plain int values manipulated with logical operators.


Without having any clue as to how hard the implementation might be, or 
how it would conflict with other uses of decorators, and without any 
concern for the exact details of the implementation, if another syntax 
would be easier, I wonder if something along the lines of new syntax 
would be preferable to a decorator, when the possibility exists for both 
flags and bitfields to be defined in the same Enum:


class SomeSpecificInt( IntBitfields ):
   COLORS( 3 ): flags = {
     RED = 1
     GREEN = 2
 BLUE = 4
 PURPLE = RED | BLUE
 },
  OPACITY( 4 ): int,
  DITHER( 4 ): values = {
    CROSSHATCH = auto(),
    SLANTED30DEGREES = auto(),
    SLANTED45DEGREES = auto(),
    SLANTED60DEGREES = auto(),
    DOTS25PERCENT = auto(),
    DOTS50PERCENT = auto(),
    DOTS75PERCENT = auto(),
    },
  }

The idea here would be that "flags" values would be unique bits that are 
in a 3-bit field, OPACITY would be an integer in a 4-bit field, and 
DITHER would be a set of unique values (only 4 defined so far, but 
allowing up to 15 values in a 4-bit field for future expansion).


To require the COLORS to have uniquely named flags, a different keyword 
would be used instead of or in addition to "flags": COLORS( 3 ): 
individual flags   [or  COLORS( 3 ): individual_flags ]


I realize this adds more complexity, and is far beyond the original 
proposal, but it also adds missing power to subdivided int values. The 
values for DITHER would be from 1-15 when extracted or cast, with the 
shifting done by the Enum implementation.


Or maybe instead of the above proposal, the decorator approach could be 
taken for individual IntFlag classes, but a new "CombinedEnum" could be 
defined to join multiple Enum classes into a single "int" with 
bitfields.  In that case, the explicit bitfield sizes given in the above 
syntax would only be used for CombinedEnum, but the member Enum classes 
that are combined would have to meet the constraint of fitting their 
values into that many bits.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/U6B74BNJ76COD4WQE7UR4IQXHU4W5F6K/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: name for new Enum decorator

[Glenn Linderman]

On 5/29/2021 6:33 PM, Nick Coghlan wrote:
On Sat, 29 May 2021, 7:27 am Ethan Furman, > wrote:


On 5/28/21 12:43 AM, Petr Viktorin wrote:
 > On 28. 05. 21 5:24, Ethan Furman wrote:

 >>      class FlagWithMasks(IntFlag):
 >>          DEFAULT = 0x0
 >>
 >>          FIRST_MASK = 0xF
 >>          FIRST_ROUND = 0x0
 >>          FIRST_CEIL = 0x1
 >>          FIRST_TRUNC = 0x2
 >>
 >>          SECOND_MASK = 0xF0
 >>          SECOND_RECALC = 0x00
 >>          SECOND_NO_RECALC = 0x10
 >>
 >>          THIRD_MASK = 0xF00
 >>          THIRD_DISCARD = 0x000
 >>          THIRD_KEEP = 0x100
 >>
 >> Here we have three flags (FIRST_MASK, SECOND_MASK, THIRD_MASK)
that are aliasing values
 >> that don't exist, but it seems intentional and not an error.
 >
 > Are you looking for a decorator for the whole Enum, or a way to
mark individual *values* as masks?

The decorator is for whole enum.  The issue is not that some
values are masks, but whether the absence of named bits
covered by the mask is an error.



If you were prepared to incur a deprecation period, then you could 
make the decorator "@allow_masks" and flip the default behaviour to 
emit Deprecation Warning for masks, eventually upgrading it to Value 
Error.


Otherwise, maybe something like "@no_masks"?


"Mask" is one typical usage for combined flags, but not the only one.  I 
would not recommend using "mask" in the name of the decorator.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LJ4NAGIKVG3PTJBBXWUVZM77S7K5JI7R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: name for new Enum decorator

[Glenn Linderman]

On 5/28/2021 12:08 AM, Michał Górny wrote:

On Thu, 2021-05-27 at 20:24 -0700, Ethan Furman wrote:

Greetings!

The Flag type in the enum module has had some improvements, but I find it 
necessary to move one of those improvements
into a decorator instead, and I'm having a hard time thinking up a name.

What is the behavior?  Well, a name in a flag type can be either canonical (it 
represents one thing), or aliased (it
represents two or more things).  To use Color as an example:

  class Color(Flag):
  RED = 1# 0001
  GREEN = 2  # 0010
  BLUE = 4   # 0100
  PURPLE = RED | BLUE# 0101
  WHITE = RED | GREEN | BLUE # 0111

The flags RED, GREEN, and BLUE are all canonical, while PURPLE and WHITE are 
aliases for certain flag combinations.  But
what if we have something like:

  class Color(Flag):
  RED = 1# 0001
  BLUE = 4   # 0100
  WHITE = 7  # 0111

As you see, WHITE is an "alias" for a value that does not exist in the Flag 
(0010, or 2).  That seems like it's probably
an error.  But what about this?

  class FlagWithMasks(IntFlag):
  DEFAULT = 0x0

  FIRST_MASK = 0xF
  FIRST_ROUND = 0x0
  FIRST_CEIL = 0x1
  FIRST_TRUNC = 0x2

  SECOND_MASK = 0xF0
  SECOND_RECALC = 0x00
  SECOND_NO_RECALC = 0x10

  THIRD_MASK = 0xF00
  THIRD_DISCARD = 0x000
  THIRD_KEEP = 0x100

Here we have three flags (FIRST_MASK, SECOND_MASK, THIRD_MASK) that are 
aliasing values that don't exist, but it seems
intentional and not an error.

So, like the enum.unique decorator that can be used when duplicate names should 
be an error, I'm adding a new decorator
to verify that a Flag has no missing aliased values that can be used when the 
programmer thinks it's appropriate... but
I have no idea what to call it.

Any nominations?


Maybe something like the following would be a good starting point:

@bitmask_require_individual_bits



@all_bits_defined

or

@unique_bits
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NE5UYXEUZ6KKHVH3Z2OX45SSWCXYEYYO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should Python typing leverage PEP 263 as a pre-processor step?

[Glenn Linderman]

On 4/19/2021 12:44 PM, Guido van Rossum wrote:
We had a similar thing at Dropbox, where `# coding: pyxl` would enable 
a preprocessor that allowed HTML embedded in the Python code. It 
translated this to function calls and string literals.


There were however several drawbacks:

- Installing the codec is a bit tricky, and if you don't have it the 
code can't run
- Other tooling won't know about the new syntax (someone went through 
heroic efforts to write a PyCharm extension for pyxl but it got stale 
very quickly)

- The codec slows down the import process
- Keeping line numbers accurate in the codec is painful
- Occasionally, when debugging mysteries, you end up needing to see 
the source code after preprocessing, which requires yet another custom 
tool


In the end the project was withdrawn and we switched to more 
mainstream templating solutions.


I suspect that many of the issues with pyxl would also plague using 
this approach as a way to customize typing syntax, and I don't think 
it would be an improvement over the status quo.



How does this "similar thing" compare to the recently announced imphook 
module?



___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QYSPZ2NNJQNEKXD2F72AFQLCX7XGO4LD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

[Glenn Linderman]

On 3/4/2021 1:41 AM, Irit Katriel wrote:



On Thu, Mar 4, 2021 at 1:38 AM Glenn Linderman <mailto:v%2bpyt...@g.nevcal.com>> wrote:


On 3/3/2021 2:49 PM, Irit Katriel via Python-Dev wrote:

That's an interesting idea.

Do you mean that one exception gets handled and the rest of the
group is reraised? Or discarded?

The value of sys.exc_info() (and the e in "except T as e:") needs
to be a single naked exception. So if there is more than one
match in the group we would need to pick one (let's say the first
in DFS order).

If we do this, then we have this situation. Before
ExceptionGroups, you got to choose which of the exceptions you
have is the most important, and you raised only that one. Now you
raise a bunch of them and the order of the except clauses in
caller's code determines which one of them counts and which ones
are discarded. What do you make of that?


You _could_ implement it as you said, but remember, you that with
this idea, you are changing how except clauses work—so instead of
making the order of the except clauses determine which one counts
most, you could instead do something else.

One alternative idea would be to take the "first in DFS order" and
see if it matches any of the except clauses, and if so, process
that one.  If not, then pick the next, and see if it matches,
until one is found that matches, and can be processed.



Or we could make it explicit:

add an optional arg to ExceptionGroup like
ExceptionGroup("eg", list_of_exceptions, singleton=None)

In the example of atexit, where currently it raises only the last 
exception from your callbacks, it will instead raise


ExceptionGroup("atexit errors", all_exceptions, singleton=last_exception)

Then except* works as before, ignoring the singleton. But except 
matches the singleton.


And there's no magic where you can be surprised about which exception 
except chose to look at.


I like explicit, and avoiding magic.

And this gives a compatibility story for outer loops that except: 
Exception, and even for others cases that are not recoded for 
ExceptionGroup handling.


And I guess what you are citing is a precedent from atexit, for raising 
the last one.


And I guess in cases other than atexit, when raising an ExceptionGroup, 
the coder of the new feature would still get more of a choice about 
which Exception is more important, rather than the coder of the except 
clauses.  One could quibble that if ValueError and IndexError were both 
part of the ExceptionGroup, that if the except clauses expected either 
might happen, and listed them in ValueError and IndexError order, that 
the intention might have been that ValueError was more interesting to 
the except clause coder, whereas if the group is raised with the 
IndexError as the singleton, that the group coder has the opposite 
intention. I think it is more likely that the except clause coder simply 
knew they were mutually exclusive and that the order of the clauses 
didn't matter.


Thinking about the above a bit, the only existing except clause sequence 
that would matter would be if both a base exception class and a derived 
class were both listed in the except clauses.  The derived exception 
class should be listed before the base exception class or it wouldn't 
get processed. So it is not clear that the order of the except clauses 
really indicates any priority of interest on the part of the except 
clause coder?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UAHFZSVVHRABZ7UQLHVLSXS53YXRIFCC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

[Glenn Linderman]

On 3/3/2021 2:49 PM, Irit Katriel via Python-Dev wrote:



On Wed, Mar 3, 2021 at 10:39 PM Greg Ewing 
mailto:greg.ew...@canterbury.ac.nz>> wrote:


On 4/03/21 5:37 am, Paul Moore wrote:
> frameworks and libraries typically have to interact with other
users'
> code, and there the contract has changed from "do what you like in
> your code and I'll cope" to "do what you like in your code as
long as
> you don't let an exception group escape, and I'll cope"... And I
have
> to change *my* code to get the old contract back.

Seems to me this whole issue would go away if the ordinary
except statement were to look inside ExceptionGroups.

In other words, the only difference between except and
except* would be that multiple except* clauses can be run,
whereas only one except clause will run (the first one that
matches something in the ExceptionGroup).

Is there any good reason not to do things that way?



That's an interesting idea.

Do you mean that one exception gets handled and the rest of the group 
is reraised? Or discarded?


The value of sys.exc_info() (and the e in "except T as e:") needs to 
be a single naked exception. So if there is more than one match in the 
group we would need to pick one (let's say the first in DFS order).


If we do this, then we have this situation. Before ExceptionGroups, 
you got to choose which of the exceptions you have is the most 
important, and you raised only that one. Now you raise a bunch of them 
and the order of the except clauses in caller's code determines which 
one of them counts and which ones are discarded. What do you make of that?


You _could_ implement it as you said, but remember, you that with this 
idea, you are changing how except clauses work—so instead of making the 
order of the except clauses determine which one counts most, you could 
instead do something else.


One alternative idea would be to take the "first in DFS order" and see 
if it matches any of the except clauses, and if so, process that one.  
If not, then pick the next, and see if it matches, until one is found 
that matches, and can be processed.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WIED3DD3HMD4WBHVO2KCGFFCXSFM6UYU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

[Glenn Linderman]

On 2/25/2021 9:40 PM, Nathaniel Smith wrote:

On Thu, Feb 25, 2021 at 2:13 PM Guido van Rossum  wrote:

So is "fail-fast if you forget to handle an ExceptionGroup" really a feature? 
(Do we call this out in the PEP?)

We may believe that "except Exception" is an abuse, but it is too common to dismiss out of hand. I 
think if some app has e.g. a main loop where they repeatedly do something that may fail in many ways (e.g. 
handle a web request), catch all errors and then just log the error and continue from the top, it's a better 
experience if it logs "ExceptionGroup:  []" than if it 
crashes.

Yeah, 'except Exception' happens a lot in the wild, and what to do
about that has been a major sticking point in the ExceptionGroup
debates all along. I wouldn't say that 'except Exception' is an abuse
even -- what do you want gunicorn to do if your buggy flask app raises
some random exception? Crash your entire web server, or log it and
attempt to keep going? (This is almost your example, but adding in the
part where gunicorn is reliable and well-respected, and that its whole
job is to invoke arbitrarily flaky code written by random users.)
Yury/I/others did discuss the idea of a
BaseExceptionGroup/ExceptionGroup split a lot, and I think the general
feeling is that it could potentially work, but feels like a
complicated and awkward hack, so no-one was super excited about it.
For a while we also had a compromise design where only
BaseExceptionGroup was built-in, but we left it non-final specifically
so asyncio could define an ExceptionsOnlyExceptionGroup.

Another somewhat-related awkward part of the API is how ExceptionGroup
and plain-old 'except' should interact *in general*. The intuition is
that if you have 'except ValueError' and you get an
'ExceptionGroup(ValueError)', then the user's code has some kind of
problem and we should probably do something? to let them know? One
idea I had was that we should raise a RuntimeError if this happens,
sort of similar to PEP 479. But I could never quite figure out how
this would help (gunicorn crashing with a RuntimeError isn't obviously
better than gunicorn crashing with an ExceptionGroup).

== NEW IDEA THAT MAYBE SOLVES BOTH PROBLEMS ==

Proposal:

- any time an unwinding ExceptionGroup encounters a traditional
try/except, then it gets replaced with a RuntimeError whose __cause__
is set to the original ExceptionGroup and whose first traceback entry
points to the offending try/except block

- CUTE BIT I ONLY JUST THOUGHT OF: this substitution happens right
*before* we start evaluating 'except' clauses for this try/except

So for example:

If an ExceptionGroup hits an 'except Exception': The ExceptionGroup is
replaced by a RuntimeError. RuntimeError is an Exception, so the
'except Exception' clause catches it. And presumably logs it or
something. This way your log contains both a notification that you
might want to switch to except* (from the RuntimeError), *along with*
the full original exception details (from the __cause__ attribute). If
it was an ExceptionGroup(KeyboardInterrupt), then it still gets caught
and that's not so great, but at least you get the RuntimeError to
point out that something has gone wrong and tell you where?

If an ExceptionGroup(ValueError) hits an 'except ValueError': it
doesn't get caught, *but* a RuntimeError keeps propagating out to tell
you you have a problem. And when that RuntimeError eventually hits the
top of your program or ends up in your webserver logs or whatever,
then the RuntimeError's traceback will point you to the 'except
ValueError' that needs to be fixed.

If you write 'except ExceptionGroup': this clause is a no-op that will
never execute, because it's impossible to still have an ExceptionGroup
when we start matching 'except' clauses. (We could additionally emit a
diagnostic if we want.)

If you write bare 'except:', or 'except BaseException': the clause
always executes (as before), but they get the RuntimeError instead of
the ExceptionGroup. If you really *wanted* the ExceptionGroup, you can
retrieve it from the __cause__ attribute. (The only case I can think
of where this would be useful is if you're writing code that has to
straddle both old and new Python versions *and* wants to do something
clever with ExceptionGroups. I think this would happen if you're
implementing Trio, or implementing a higher-level backport library for
catching ExceptionGroups, something like that. So this only applies to
like half a dozen users total, but they are important users :-).)

-n

I wondered why an ExceptionGroup couldn't just be an Exception. This 
effectively makes it so (via wrapping).


So then why do you need  except*  at all?  Only to catch unwrapped 
ExceptionGroup before it gets wrapped?


So why not write except ExceptionGroup, and let it catch unwrapped 
ExceptionGroup?


That "CUTE BIT" could be done only when hitting an except chain that 
doesn't include an except ExceptionGroup.


Nope, I didn't read the PEP, and don't 

[Python-Dev] Re: __init_subclass__ and metaclasses

[Glenn Linderman]

On 12/29/2020 10:00 PM, Guido van Rossum wrote:

I need to think about this more.


Both techniques could coexist.

Technically the class *is* created at the point `__init_subclass__` is 
called. What the metaclass does after that point is embellishment.


Most metaclasses will have sufficient control for their needs because 
they can manipulate the contents of the namespace that's passed to 
type_new() -- I understand that doesn't work for Enum because the 
instances have a reference to the class in them.


I suspect the classes that you claim are "buggy" are fine in practice, 
even if you can construct a counterexample.


All in all I just worry about the backward compatibility here: one 
could easily imagine a metaclass whose `__new__` relies on the effect 
of the `__init_subclass__` called by type_new(), and moving the call 
would break that. So it's probably 6 bugs fixed, half a dozen new bugs 
created. In such a case, the status quo should win.


--Guido

On Tue, Dec 29, 2020 at 10:44 AM Ethan Furman > wrote:


On 12/29/20 8:59 AM, Guido van Rossum wrote:
> On Mon, Dec 28, 2020 at 10:24 PM Ethan Furman wrote:

>> The `__init_subclass__` and `__set_name__` protocols are
intended to be run before a new type is finished, but creating
>> a new type has three major steps:
>>
>> - `__prepare__` to get the namespace
>> - `__new__` to get the memory and data structures
>> - `__init__` for any final polishing
>>
>> We can easily move the calls from `type_new()` to `type_init`.
>
> No, we can't. There is a window where the subclass is
initialized after `typing_new()` returned before `__init__`
> starts, and you propose to move the subclass after that window.
There may be code that depends on the class being
> initialized at that point, and you will break that code.

True, there will be a few custom metaclasses that need to move
some code from their `__new__` to `__init__` instead, and
a few that need to add an `__init__` to consume any keyword
arguments that don't need to get passed to
`__init_subclass__`.  That seems like a small price to pay to be
able to write custom metaclasses that are able to fully
participate in the `__set_name__` and `__init_subclass__` protocols.

Just in the stdlib we have two custom metaclasses that would start
working correctly with this change (for the
`__init_subclass__` case).

> Honestly I disapprove of the shenanigans you're committing in
enum.py, and if those are in in the 3.10 (master) branch I
> recommend that you take them out. It just looks too fragile and
obscure.

Trust me, I don't like them either.  But I like even less that a
custom `__init_subclass__` for an Enum would fail if it
tried to do anything with the members.  That's one of the reasons
why I would like to see this fix put in (the
shenanigans would be unnecessary then).

 From the point of view of a metaclass author, the current
behavior feels buggy.  In theory, `__init_subclass__` and
`__set_name__` are supposed to be called after a class is created,
and yet they are being called somewhere in the middle
of my metaclass' `__new__`.

Looked at another way, `__init_subclass__` should be receiving a
`cls` that is ready for use, i.e. done and fully
constructed and complete. But if that class is being created by a
custom metaclass, then the thing that is being given
to `__init_subclass__` could easily be only partial.

--
~Ethan~
___
Python-Dev mailing list -- python-dev@python.org

To unsubscribe send an email to python-dev-le...@python.org

https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at

https://mail.python.org/archives/list/python-dev@python.org/message/7WPNGL267FDGN6WWCHZQUAXWVBTUJOMN/


Code of Conduct: http://python.org/psf/codeofconduct/




--
--Guido van Rossum (python.org/~guido )
/Pronouns: he/him //(why is my pronoun here?)/ 



___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AD2ORFW6VCYQNIKCW2DZ3F5KVADDYYZC/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-Dev 

[Python-Dev] Re: Words rather than sigils in Structural Pattern Matching

[Glenn Linderman]

On 11/21/2020 9:47 AM, David Mertz wrote:
So in my mind, if I had the choice, it is a decision between a sigil 
and a word to indicate "no, really use this name as a value!" I like a 
word better, but none of the current keywords really make sense, so it 
would need to be a new word. I suggested "value", but another word 
might be better.


NOT_FOUND = 404
match var:
    case 200:
    print("OK")
    case (NOT_FOUND):
    print("Not Found")
    case other_code:
    print("Other code", other_code )

Don't () already indicate an expression to be evaluated?

It's only one character longer than the early . suggestion, and already 
has the desired meaning of "calculated value" not "bound variable".

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EQ2DSQ3Y4FBZQGW36NKCAAABOUSPZVWH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 642 v2: Explicit constraint patterns *without* question marks in the syntax

[Glenn Linderman]

On 11/15/2020 11:18 PM, Stephen J. Turnbull wrote:

Jim J. Jewett writes:

  > What advantage can there be in re-using syntax to mean something
  > opposite to what it normally does?

In math, it allows us to write "solve c = f(x) for x".  That doesn't
mean "bind c to the value f(x)", it means exactly the opposite.  No
problem here, I suppose.  So

 match p:
 case Point(x=a, y=b):

is a way of saying "solve p = Point(x=a, y=b) for a and b".

I understand your distaste for this syntax, but I see no problem
in principle.  It's a problem of programmer habits and the evident
inconsistency with forms like "case Point(x=1, y=2)".  This is
especially true when a or b is already bound.
Indeed. Mathematics never came up with separate symbols for equality and 
assignment. Neither did early programming languages. That was a source 
of confusion for many programmers.


Then someone invented == for equality comparison, and = for assignment, 
or alternately := for assignment and = for equality comparison.  Python 
now has the first three, many other languages only have one of the pairs.


This reduced the confusion and resulting bugs, although some still 
happen. Linters help avoid them, and even some compilers emit warnings 
for possibly misused operators of those pairs.


In mathematics, equality might be written a = b or b = a, transitively 
irrelevant.


But I have never seen assignment in mathematics be anything but var = 
value, as in
"let x = 3"  or "let x = ". Maybe I'm just 
uninformed regarding current practice, but I do have a degree in 
mathematics, albeit somewhat dated at this point.


So I also have a distaste for any syntax that puts an assignment target 
on the right, it seems like a step toward reintroducing confusion and 
bugs. The pattern match operator seems complex enough without 
(re-)introducing confusion regarding assignment operators.


In your example "solve c = f(x) for x", that is not an assignment 
operator, but an equality comparison operator in an equation. Although I 
know a few exist, I'm not familiar with programs that do symbolic 
algebra, to know how they notate such instructions: most programming 
language do not do symbolic manipulations.  But note that your example 
is wordier than "let c = f(x)" which would be an assignment operation. 
And note that it _needs to be wordier_ to convey the proper mathematical 
semantics. At least, it did back when I was in math classes.


Just because something is syntactically unambiguous, doesn't mean it 
makes a good user interface.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/A7OGGP3YKYD32LYCYC3NWNUWC7ICUPIX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 642 v2: Explicit constraint patterns *without* question marks in the syntax

[Glenn Linderman]

On 11/13/2020 1:48 PM, Joao S. O. Bueno wrote:



On Fri, 13 Nov 2020 at 17:36, Jim J. Jewett > wrote:


I *hope* this was a typo!  If

    case Point(x=a, y=b):

assigns to a and b (instead of x and y, as in a normal call), then
that is ... going to be very easy for me to forget, and to miss
even when I'm aware of it.


No typo -  this is _precisely_what the main proposal on PEPs 634, 635 
and 636 is suggesting, and tha PEP 642 is trying to avoid.


And the claim is, that there are people that think this is a good idea❓❓ 
And are actually proposing to add it to Python❓



Here it is working as is  on my Python with PEP 622 build:

```
Python 3.10.0a0 (heads/patma:1ba56a003b, Aug  6 2020, 02:00:58)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.17.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: fromdataclassesimportdataclass

In [2]: @dataclass
  ...: classPoint:
  ...: x: float
  ...: y: float
  ...:

In [3]: defget_coords(p):
  ...: match p:
  ...: case Point(x=a, y=b):
  ...: returna, b
  ...: case _:
  ...: raiseTypeError()
  ...:

In [4]: get_coords(Point(3, 4))
Out[4]: (3, 4)

```


-jJ



___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UVOJJF37B5Q5IQJWV3YKBOZA23BY6XI5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 642: Constraint Pattern Syntax for Structural Pattern Matching

[Glenn Linderman]

On 11/2/2020 1:42 PM, Guido van Rossum wrote:
But we feel that `case x, x` can easily be misunderstood as "a tuple 
of two equal values"


So what _is_ the syntax for "a tuple of two equal values" ?

case x, ?x:  # comes to mind (not that it is in the PEP :))
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GPSLYBPMXRFQHUST7CYTI6LNJY7D25EH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622 version 2 (Structural Pattern Matching)

[Glenn Linderman]

On 7/16/2020 10:00 AM, Rhodri James wrote:

On 16/07/2020 17:37, Steve Holden wrote:

While I understand the point of view that says that match ... :
should encapsulate a sequence of indented suites, it seems to me that
match/case/case/.../else has a natural affinity with
try/except/except/.../finally/else, and nobody seems to think that the
excepts should be indented. Or the finally. And naturally the match/else
case are at the same indentation level, just as for/else, while/else and
try/finally. So why, exactly, should case be indented?


My take on the difference would be that "try" tries out a suite, while 
"match" matches an expression.  If we did:


    match:
    
    case :
    

then having an indented section which must be a single expression 
would be unique in Python syntax.  I could easily see people being 
confused when the slew of statements they would inevitably decide they 
must be able to put there, and soon we'd have cats and dogs living 
together and the downfall of civilisation as we know it.


Alternatively:

    match :
    case :
    

would be the one place in Python where you end a line with a colon and 
*don't* indent the following line.  Writers of simple formatters and 
the like (such as Python-mode in Emacs) would curse your name, etc, etc.



My apologies for being a Bear of Very Little Brain.


Nah, don't apologise.  This is one of those things that everyone has 
opinions on, because there doesn't seem to be an obvious Right Answer.




Speaking of things unique in Python syntax, isn't double-indention of a 
single control flow structure introduced by match also unique?

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/E26GHBGQCGROPUJZBRY66GVKDWOQT4ZX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622 version 2 (Structural Pattern Matching)

[Glenn Linderman]

On 7/11/2020 10:36 AM, Jim J. Jewett wrote:

Glenn Linderman wrote:

On 7/10/2020 3:15 AM, Gustavo Carneiro wrote:

...

Therefore, I posit that the style of try...except indentation only
works where the number of cases is small.
But for the case of pattern matching, I expect the number of cases to
be matched to be a lot higher than exception handling cases.  Having
cases to be matched be indented is, IMHO, a nice visual cue to help
the reader understand where the pattern matching block ends.

Actually, the current  if elseif elseif elseif else, used now because
Python has no switch/match/case, has exactly the same issue as you
describe as a problem with try if there were more cases... and if often
has more cases, just like match will.

True


So your concern seems nebulous. You may have wished for extra
indentation but it is simple to get more indentation: use 8 spaces
instead of 4. So if you really wanted it, you could have had it.

Not so true.


It is true, the way I meant it, but not the way you (mis-)interpreted it.


 if a: ...
 elif b: ...
 elif c: ...
 else:

is not valid.  In practice, I interpret wanting to indent at a place that 
doesn't require it as a code smell suggesting I should try to break out a 
helper function, but ... it does happen.


if  a:
    ...
elif b:
    ...
elif c:
    ...
else:
    ...

is perfectly valid.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IEZ5PTLDDGFTT6763FUH4MNXNXFNWAR2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622 version 2 (Structural Pattern Matching)

[Glenn Linderman]

On 7/10/2020 3:15 AM, Gustavo Carneiro wrote:



On Fri, 10 Jul 2020 at 10:33, Glenn Linderman <mailto:v%2bpyt...@g.nevcal.com>> wrote:


On 7/10/2020 1:21 AM, Stefano Borini wrote:
> Just my 2 cents, I find it kind of annoying that the whole structure
> requires two levels of indentation to actually reach the operational
> code.
> This would be a first in python.
>
> I would prefer an option akin to if elif elif else where each
block is
> only one level deep.
Me too.

That would also sidestep the dilemma of whether else: (if
implemented)
should be indented like case: or like match: because they would be
the same.

match:
 t
case ("rect", real, imag):
 return complex(real, imag)
case ("polar", r, phi):
 return complex( r* cos(phi), r*sin(phi)
else:
 return None

but it does make the match: block not a statement group, which was
disturbing to some.

On the other hand, this has a correspondence to:

try:
  throw expression
except (type of expression) as exc1:
  blah blah1
except (another type) as exc2:
 blah blah2
else:
 blah blah3


The problem of the try...except structure, with less indentation, is 
that, yes, it is OK for exceptions because normally you have 2 or 3 
`except XXX` clauses, therefore it is usually easy to follow, if the 
number of vertical lines in the entire block of try-catch is low enough.


But I have had cases with catching many exception types, each with its 
own block of 4 or 5 lines, adding up to a block of try-excepts that 
doesn't even fit in a single window of my editor.  In that case, I 
always have wished for except clauses to be extra indented, to more 
easily distinguish where the try..except block ends.


Therefore, I posit that the style of try...except indentation only 
works where the number of cases is small.


But for the case of pattern matching, I expect the number of cases to 
be matched to be a lot higher than exception handling cases.  Having 
cases to be matched be indented is, IMHO, a nice visual cue to help 
the reader understand where the pattern matching block ends.


Actually, the current  if elseif elseif elseif else, used now because 
Python has no switch/match/case, has exactly the same issue as you 
describe as a problem with try if there were more cases... and if often 
has more cases, just like match will.


So your concern seems nebulous. You may have wished for extra 
indentation but it is simple to get more indentation: use 8 spaces 
instead of 4. So if you really wanted it, you could have had it. It is 
much harder to get less indentation when the language structures 
prescribe it.



In fact, one _could_ wrap this whole feature into the try:
syntax... the
match statement would be tried, and the cases would be special
types of
exception handlers:

try:
 match expression
case ("rect", real, imag):
 return complex(real, imag)
case ("polar", r, phi):
 return complex( r* cos(phi), r*sin(phi)
else:
 return None

If the expression could fail to be calculated, one could have a
mix of
except clauses also to catch those, rather than needing to wrap the
whole match expression in a separate try to handle that case
[making the
nesting even deeper :( ]

There might even be a use for using case clauses to extend "normal"
exception handling, where the exception object could be tested for
its
content as well as its class to have different handling.

try:
 raise Exception("msg", 35, things)
case Exception( x, "widgets"):
 blah blah 1
case Exception( x, "characters"):
 blah blah 2
else:
 blah blah 3

In this not-fully-thought-through scenario, maybe the keyword match
isn't even needed: "raise expression" could do the job, or they
could be
aliases to signify intent.

In other words, a match expression would always "fail". The only
mismatch here is that it points out the difference between
try-else and
match-else: try-else is executed if there is no failure, but if match
always fails, else would never be appropriate, and case _: would be.

In any case, it does seem there is a strong correlation between match
processing and try processing, that I didn't see during other
discussions of the possible structural similarities. "match 3 / 0:"
would clearly need to be wrapped in a try:

try:
 match x / y:
  case 43:
    print("wow, it is 43")
  case 22:
    print("22 seemed less likely than 43 for some reason")
 case _:
   print("You ge

[Python-Dev] Re: PEP 622 version 2 (Structural Pattern Matching)

[Glenn Linderman]

On 7/10/2020 1:21 AM, Stefano Borini wrote:

Just my 2 cents, I find it kind of annoying that the whole structure
requires two levels of indentation to actually reach the operational
code.
This would be a first in python.

I would prefer an option akin to if elif elif else where each block is
only one level deep.

Me too.

That would also sidestep the dilemma of whether else: (if implemented) 
should be indented like case: or like match: because they would be the same.


match:
    t
case ("rect", real, imag):
    return complex(real, imag)
case ("polar", r, phi):
    return complex( r* cos(phi), r*sin(phi)
else:
    return None

but it does make the match: block not a statement group, which was 
disturbing to some.


On the other hand, this has a correspondence to:

try:
 throw expression
except (type of expression) as exc1:
 blah blah1
except (another type) as exc2:
    blah blah2
else:
    blah blah3

In fact, one _could_ wrap this whole feature into the try: syntax... the 
match statement would be tried, and the cases would be special types of 
exception handlers:


try:
    match expression
case ("rect", real, imag):
    return complex(real, imag)
case ("polar", r, phi):
    return complex( r* cos(phi), r*sin(phi)
else:
    return None

If the expression could fail to be calculated, one could have a mix of 
except clauses also to catch those, rather than needing to wrap the 
whole match expression in a separate try to handle that case [making the 
nesting even deeper :( ]


There might even be a use for using case clauses to extend "normal" 
exception handling, where the exception object could be tested for its 
content as well as its class to have different handling.


try:
    raise Exception("msg", 35, things)
case Exception( x, "widgets"):
    blah blah 1
case Exception( x, "characters"):
    blah blah 2
else:
    blah blah 3

In this not-fully-thought-through scenario, maybe the keyword match 
isn't even needed: "raise expression" could do the job, or they could be 
aliases to signify intent.


In other words, a match expression would always "fail". The only 
mismatch here is that it points out the difference between try-else and 
match-else: try-else is executed if there is no failure, but if match 
always fails, else would never be appropriate, and case _: would be.


In any case, it does seem there is a strong correlation between match 
processing and try processing, that I didn't see during other 
discussions of the possible structural similarities. "match 3 / 0:" 
would clearly need to be wrapped in a try:


try:
    match x / y:
 case 43:
   print("wow, it is 43")
 case 22:
   print("22 seemed less likely than 43 for some reason")
    case _:
  print("You get what you get")
except ZeroDivisionError as exc:
    print(f"But sometimes you get an exception {exc}")

or:

try:
    raise x / y
case 43:
    print("wow, it is 43")
case 22:
    print("22 seemed less likely than 43 for some reason")
case exc := ZeroDivisionError:
    print(f"But sometimes you get an exception: {exc}")
case _:
    print("You get what you get")
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GDP2KKB3SUWQZRSNTR5N36LXZ6HDS2QL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Recent PEP-8 change

[Glenn Linderman]

On 7/2/2020 7:39 AM, Tim Peters wrote:

Then again, we're talking about humans.  There's nothing you can do -
or refrain from doing - that won't mortally offend someone:-)

This is the truest thing spoken in this whole thread.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SLGATIV7XP4C4SQHSALP7LNQ5CQMNIP6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Plan to remove Py_UNICODE APis except PEP 623.

[Glenn Linderman]

On 7/1/2020 1:20 PM, M.-A. Lemburg wrote:

As an example application, think of a database module which provides
the Unicode data as Py_UNICODE buffer. You want to write this as UTF-8
data to a file or a socket, so you have the PyUnicode_EncodeUTF8() API
decode this for you into a bytes object which you can then write out
using the Python C APIs for this.
But based on Victor's survey of usages in Python extensions, which found 
few to no uses of these APIs, it would seem that hypothetical 
applications are insufficient to justify the continued provision and 
maintenance of these APIs.


After all, Python extensions are written as a way to interface "other 
stuff" to Python, and converting data to/from Python objects seems far 
more likely than converting data from one non-Python format to a 
different non-Python format. Not that such applications couldn't be 
written as Python extensions, but ... are they? ... and why?


A rich interface is nice, but an unused interface is a burden.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MA7U7A6OCO46TEWXRKHGDRBYH5FMWJUP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Glenn Linderman]

On 6/24/2020 1:49 PM, Tim Peters wrote:

".NAME" grated at first, but extends the idea that dotted names are
always constant value patterns to "if and only if". So it has mnemonic
value. When context alone can't distinguish whether a name is meant as
(in effect) an lvalue or an rvalue, no syntax decorations can prevent
coding errors. Names in destructuring constructs are overwhelmingly
intended as lvalues, so adding extra cruft to say "no, I meant rvalue"
is the pragmatic choice.

This is just a bunch of words to me, without meaning.

I'd like to understand it.

What do you mean by "the idea that dotted names are always constant 
value patterns"?


What do you mean by 'extends (the above) to "if and only if" '?

As a result of not understanding the above, I see no mnemonic value.

My understanding of the "." as proposed is that it is optional, except 
in cases where it would be ambiguous... seems like it would be better if 
it were required for one case or the other, so that there would be no 
need to determine whether or not it is ambiguous by the surrounding 
state/code/declarations.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LL5KF2DL4VIZO44FPZRZW27PM55HCJAU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 554 for 3.9 or 3.10?

[Glenn Linderman]

On 4/21/2020 10:26 PM, Greg Ewing wrote:

And if I understand correctly, you won't get any nice "This
module does not support subinterpreters" exception if you
import an incompatible module -- just an obscure crash,
probably of the core-dumping variety. 


This sounds fixable: modules that support subinterpreters should set a 
flag saying so, and the either the load of a non-flagged module when 
subinterpreters are in use, or the initiation of a subinterpreter when a 
non-flagged module has been loaded, should raise.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/H3TCU43YUSUMYUMWBA6PFZZJ7AG5FZCD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Glenn Linderman]

On 4/15/2020 2:57 PM, Ivan Pozdeev via Python-Dev wrote:


On 16.04.2020 0:34, Glenn Linderman wrote:

On 4/15/2020 12:47 PM, Ivan Pozdeev via Python-Dev wrote:
When writing a proof-of-concept implementation, however, I bumped 
into the need to distinguish which of the child objects are 
containers (thus need to be wrapped as well) and which are the 
leaves (thus need to be returned as is). I guess this is going to be 
application-specific. And the same problem will arise with any 
implementation, not only a wrapper.


Do the child objects truly need to be wrapped, or just accessed?


They need to be wrapped on the fly, to support nested  ".key" accesses.


That sounds like an implementation detail... that is one possible 
implementation.

The parent object could alternately look inside, it seems.




Thanks for your comments though, they inspired a thought.

The problem with the glom syntax versus the dotted syntax is that the 
minimal syntax is bulky.


obj.abc.def.ghi versus   glom( obj, 'abc.def.ghi')

The problem with attribute syntax is the conflict with regular 
attributes, and the limitation of valid identifier characters. Glom 
probably doesn't have these problems.


"Glom syntax" still excludes the delimiter, whatever it is, from use 
in keys. So it's still a further limitation compared to the JSON spec.


That could be handled with escaping, when necessary.





So my new thought is that we need a new syntax.  The minimal existing 
syntax might be


obj._('abc.def.ghi')     or maybe   obj['abc.def.ghi']

although the latter would conflict with regular dict lookups, but obj 
isn't a regular dict, so that might not matter.



A new syntax might be:

obj.'abc.def.ghi'


In any of these syntaxes, one could enhance it to use variables in 
some spots similarly to f-strings (but from the beginning, so no 'f' 
would be required)


foo = 'def'
obj.'abc.{foo}.ghi'

would access the same item as previous examples.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3A5V62LY3DAPAREZMB7MVRYX4432NR7F/

Code of Conduct: http://python.org/psf/codeofconduct/
--
Regards,
Ivan

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/76BY73UZRU2QUTEMDQDD6J4TIVSOWKJ7/

Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BG3MJYLHGS4E4QDRRPSOJHXBEBPSHRQC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Glenn Linderman]

On 4/15/2020 12:47 PM, Ivan Pozdeev via Python-Dev wrote:
When writing a proof-of-concept implementation, however, I bumped into 
the need to distinguish which of the child objects are containers 
(thus need to be wrapped as well) and which are the leaves (thus need 
to be returned as is). I guess this is going to be 
application-specific. And the same problem will arise with any 
implementation, not only a wrapper.


Do the child objects truly need to be wrapped, or just accessed?

Thanks for your comments though, they inspired a thought.

The problem with the glom syntax versus the dotted syntax is that the 
minimal syntax is bulky.


obj.abc.def.ghi versus   glom( obj, 'abc.def.ghi')

The problem with attribute syntax is the conflict with regular 
attributes, and the limitation of valid identifier characters.  Glom 
probably doesn't have these problems.


So my new thought is that we need a new syntax.  The minimal existing 
syntax might be


obj._('abc.def.ghi')     or maybe   obj['abc.def.ghi']

although the latter would conflict with regular dict lookups, but obj 
isn't a regular dict, so that might not matter.



A new syntax might be:

obj.'abc.def.ghi'


In any of these syntaxes, one could enhance it to use variables in some 
spots similarly to f-strings (but from the beginning, so no 'f' would be 
required)


foo = 'def'
obj.'abc.{foo}.ghi'

would access the same item as previous examples.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3A5V62LY3DAPAREZMB7MVRYX4432NR7F/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Glenn Linderman]

On 4/14/2020 10:09 PM, Cameron Simpson wrote:

On 14Apr2020 21:25, Guido van Rossum  wrote:

On Tue, Apr 14, 2020 at 9:08 PM Raymond Hettinger <
raymond.hettin...@gmail.com> wrote:

[GvR]
> We should not try to import JavaScript's object model into Python.

Yes, I get that.  Just want to point-out that working with heavily 
nested

dictionaries (typical for JSON) is no fun with square brackets and
quotation marks.


Yeah, I get that too. So maybe this should be limited to JSON? Could the
stdlib json library be made to return objects that support this 
(maybe with

an option)?


I find this feels slightly special purpose. Though admirably 
restrained.  I think I dislike the need to detour thorough the json 
module to get such a feature.


Like many others, I recently implemented one of these 
__getattr__+__getitem__ SimpleNamespaces.  I'm hacking on some 
mappings which map dotted-names to values.  So the natural 
implementation is dicts or dict subclasses.  But I'm also feeding 
these to format strings, so I want to write:


 "Hi {dotted.name}"

so I've made a nesting of SimpleNamespace so that I can use nice 
dotted a.b.c.d type names in the format string. And because I'm using 
.format_map, the top level namespace also supports __getitem__.


Now, my dict subclass has a .ns() method returning one of the above 
SimpleNamespace subclasses, but I can readily imagine a utility 
function in collection that made such a thing.


Restricting that to only work via some contrived path through the JSON 
module seems... clunky.


Cheers,
Cameron Simpson 


Interesting comments. So .ns allows you to convert a nested dict to a 
dotted one "on demand" but the initial conversion only needs to do the 
top level object?


That sounds similar to the glom that Nathaniel brought to the attention 
of this thread, which I hadn't found before.  But glom requires bulkier 
syntax. When I read his example, I thought to myself "What if one could 
write his 'glom(json_obj, "a.b.c")' example as  json_obj['a.b.c']   or 
better as non_json_obj.a.b.c ?  I can imagine an implementation of my 
first "what-if" fully in the top-level object, but not of the second 
(but my internals imagination is probably limited).


Indeed, many of my use-cases are non-json, although a bunch are json also.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JYPDZ4I65DWTS6F7FBNE2WGGG2DXZV6N/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Glenn Linderman]

On 4/14/2020 9:25 PM, Guido van Rossum wrote:
On Tue, Apr 14, 2020 at 9:08 PM Raymond Hettinger 
mailto:raymond.hettin...@gmail.com>> wrote:


[GvR]
> We should not try to import JavaScript's object model into Python.

Yes, I get that.  Just want to point-out that working with heavily
nested dictionaries (typical for JSON) is no fun with square
brackets and quotation marks.


Yeah, I get that too. So maybe this should be limited to JSON? Could 
the stdlib json library be made to return objects that support this 
(maybe with an option)?


I replied in the issue, not noticing the discussion was happening here 
after getting interrupted with something else.


But I use my implementation (uploaded to the issue) for many things 
besides JSON. Such a feature is just too practical not to be Pythonic.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IGM424TB3L7FR536PFJZN2IYILGGA6YP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are PyObject_RichCompareBool shortcuts part of Python or just CPython quirks?

[Glenn Linderman]

On 2/3/2020 6:21 PM, Chris Angelico wrote:


Hmm, true, although that's equivalent only in one specific situation.
In mathematics, "congruent" means that two things are functionally
equivalent (eg triangles with the same length sides; in programming
terms we'd probably say that two such triangles would be "equal" but
not identical), even if there's a specific context for such
equivalence, such as stating that 12,345 is congruent to 11 modulo 7,
because the remainders 12345%7 and 11%7 are both 4. So maybe
"congruent" could be used for this concept?


Congruent is different objects with the same characteristics, whereas 
identical is far stronger: same objects.


But the reason  <=  and   >=  were invented was to avoid saying

a < b  or  a  == b and   a > b  or   a == b

It is just a shorthand.

So just invent   is==   as shorthand for   a is b  or  a == b.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2ESFOJXCSMVBBKY5L2FJKLDWYNH4POUJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Parameters of str(), bytes() and bytearray()

[Glenn Linderman]

On 12/16/2019 12:05 AM, Kyle Stanley wrote:

Chris Angelico wrote:
> ANY object can be passed to str() in order to get some sort of valid
> printable form. The awkwardness comes from the fact that str()
> performs double duty - it's both "give me a printable form of this
> object" and "decode these bytes into text".

While it does make sense for str() to be able to give some form of 
printable form for any object, I suppose that I just don't consider 
something like this: "b'\\xc3\\xa1'" to be overly useful, at least for 
any practical purposes. Can anyone think of a situation where you 
would want a string representation of a bytes object instead of 
decoding it?



Binary data
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BLYF3OAG6LLVBRJ4J6A6QSGC7KFKBX7I/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [RELEASE] Python 3.8.0 is now available

[Glenn Linderman]

On 10/15/2019 11:03 AM, MRAB wrote:

On 2019-10-14 21:23, Łukasz Langa wrote:
On behalf of the Python development community and the Python 3.8 
release team, I’m pleased to announce *the availability of Python 
3.8.0*.



[snip]
I've installed pywin32 on Python 3.8, but when I try to import 
win32clipboard it says it can't find it:


Python 3.8.0 (tags/v3.8.0:fa919fd, Oct 14 2019, 19:37:50) [MSC v.1916 
64 bit (AMD64)] on win32

Type "help", "copyright", "credits" or "license" for more information.
>>> import win32
>>> import win32clipboard
Traceback (most recent call last):
  File "", line 1, in 
ImportError: DLL load failed while importing win32clipboard: The 
specified module could not be found.

>>>

Does anyone else have this problem?


Yes, and also win32gui (of pywin32) gets the same error.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3OCA42T6Q7JEDKNE55YW46MSNDAXIK6M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [RELEASE] Python 3.8.0 is now available

[Glenn Linderman]

On 10/14/2019 9:26 PM, Glenn Linderman wrote:

On 10/14/2019 1:23 PM, Łukasz Langa wrote:

On behalf of the Python development community and the Python 3.8 release team, 
I’m pleased to announce the availability of Python 3.8.0.

I look forward to using Python 3.8.0.

However, having installed it, I then needed to install brotli, so I 
ran pip install brotli, and that worked, but I was very surprised to 
get told:


You are using pip version 18.1, however version 19.3 is available.
You should consider upgrading via the 'python -m pip install --upgrade 
pip' command.


The upgrade worked, but why would the newest, just-released version of 
Python not include the newest version of pip?


Glenn


Hmm. "worked" above only means "completed without reporting errors".

Excuse the noise, I accidentally ran Python3.6\Scripts\pip.exe.  But now 
running the one from Python3.8 the error message from pip references pip 
version 19.2.3 as the "old version", so it is still true that Python 3.8 
doesn't seem to include the latest pip. But with the numbers being 
closer, I suppose that means that pip had a release after the Python 3.8 
code freeze.


And sadly, I'm too fast at trying to install brotli for 3.8: it 
apparently doesn't have a wheel yet, so tried to compile from source, 
and couldn't find a C compiler on my machine.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CXEP63YLKKWLCNYASOEHJTRB72LU3Q3U/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python for Windows (python-3.7.4.exe) location confusing

[Glenn Linderman]

On 9/9/2019 2:48 AM, Steve Dower wrote:
User with administrative privileges are by implication better able to 
handle decisions such as this. If they are not, they should not be 
administrating a machine. 
Most home machines are administered by people that should not be 
"administrating" a machine.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2LQT5VKDV5PS4DL4VURQ5JKPZLKXVRYJ/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/15/2019 4:17 AM, Petr Viktorin wrote:

On 8/15/19 10:40 AM, Greg Ewing wrote:

If we want a truly raw string format that allows all characters,
including any kind of quote, we could take a tip from Fortran:

 s = 31HThis is a "totally raw" string!


Or from Rust:

let s = r"Here's a raw string";
let s = r#"Here's a raw string with "quotes" in it"#;
let s = r##"Here's r#"the raw string syntax"# in raw string"##;
let s = r###"and here's a '##"' as well"###;


Indeed, Fortran has raw strings, but comes with the disadvantage of 
having to count characters. This is poor form when edits want to change 
the length of the string, although it might be advantageous if the 
string must fit into a certain fixed-width on a line printer. Let's not 
go there.


Without reading the Rust spec, but from your examples, it seems that 
Rust has borrowed concepts from Perl's q and qq operators, both of which 
allowed specification of any non-alphanumeric character as the 
delimiter. Not sure if that included Unicode characters (certainly not 
in the early days before Unicode support was added), but it did have a 
special case for paired characters such as <> [] {} to allow those pairs 
to be used as delimiters, and still allow properly nested instances of 
themselves inside the string.


It looks like Rust might only allow #, but any number of them, to 
delimit raw strings. This is sufficient, but for overly complex raw 
strings containing lots of # character sequences, it could get 
cumbersome, and starts to border on the problems of the Fortran 
solution, where character counting is an issue, whereas the choice of an 
alternative character or character sequence would result in a simpler 
syntax.


I don't know if Rust permits implicit string concatenation, but a quick 
search convinces me it doesn't.


The combination of Python's triple-quote string literal, together with 
implicit concatenation, is a powerful way to deal with extremely complex 
string literals, although it does require breaking them into pieces 
occasionally, mostly when including a string describing the triple-quote 
syntax. Note that regex searching for triple-quotes can use "{3} or '{3} 
to avoid the need to embed triple-quotes in the regex.


Perl's "choice of delimiter" syntax is maybe a bit more convenient 
sometimes, but makes parsing of long strings mentally exhausting 
(although it is quick for the interpreter), due to needing to remember 
what character is being used as the delimiter.


My proposal isn't intended to change the overall flavor of Python's 
string syntax, just to regularize and simplify it, while allowing 
additional escapes and other extensions to be added in the future, 
without backward-compatibility issues.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PUY3DUQKAPQEQLPRZCZX2NAD4Z2KPIJW/

[Python-Dev] Re: An f-string issue [Was: Re: Re: What to do about invalid escape sequences]

[Glenn Linderman]

On 8/14/2019 8:09 AM, Random832 wrote:

On Sat, Aug 10, 2019, at 19:54, Glenn Linderman wrote:

Because of the "invalid escape sequence" and "raw string" discussion,
when looking at the documentation, I also noticed the following
description for f-strings:


Escape sequences are decoded like in ordinary string literals (except when a 
literal is also marked as a raw string). After decoding, the grammar for the 
contents of the string is: followed by lots of stuff, followed by
Backslashes are not allowed in format expressions and will raise an

error: f"newline: {ord('\n')}"  # raises SyntaxError
  What I don't understand is how, if f-strings are processed AS
DESCRIBED, how the \n is ever seen by the format expression.

  The description is that they are first decoded like ordinary strings,
and then parsed for the internal grammar containing {} expressions to
be expanded. If that were true, the \n in the above example would
already be a newline character, and the parsing of the format
expression would not see the backslash. And if it were true, that would
actually be far more useful for this situation.

  So given that it is not true, why not? And why go to the extra work of
prohibiting \ in the format expressions?

AIUI there were strong objections to the "AS DESCRIBED" process (which would 
require almost all valid uses of backslashes inside to be doubled, and would incidentally 
leave your example *still* a syntax error), and disallowing backslashes is a way to 
pretend that it doesn't work that way and leave open the possibility of changing how it 
works in the future without breaking compatibility.

The only dubious benefit to the described process with backslashes allowed would be that f-strings (or other strings, 
in the innermost level) could be infinitely nested as f'{f\'{f\\\'{...}\\\'}\'}', rather than being hard-limited to 
four levels as f'''{f"""{f'{"..."}'}"""}'''


Sure. I am just pointing out (and did so in the issue I created for 
documentation as well), that the documentation does not currently 
correctly describe the implemenation, which is misleading to the user.


While I have opinions on how things could work better, my even stronger 
opinion is that documentation should *accurately* describe how things 
work, even if it how it works is more complex than it should be.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/B2HI27XRCA43GVV2D2MF5IZOUX5NG2PW/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/14/2019 8:02 AM, Random832 wrote:

On Mon, Aug 12, 2019, at 15:15, Terry Reedy wrote:

Please no more combinations. The presence of both legal and illegal
combinations is already a mild nightmare for processing and testing.
idlelib.colorizer has the following re to detest legal combinations

  stringprefix = r"(?i:r|u|f|fr|rf|b|br|rb)?"

More advanced syntax highlighting editors have to handle each string type separately 
anyway, because they highlight (valid) backslash-escapes and f-string formatters. 
The proposed 'v-string' type would need separate handling even in a simplistic 
editor like IDLE, because it's different at the basic level of \" not ending 
the string (whereas, for better or worse, all current string types have exactly the 
same rules for how to find the end delimiter)
I had to read this several times, and then only after reading Eric's 
reply, it finally hit me that what you are saying is that \" doesn't end 
the string in any other form of string, but that sequence would end a 
v-string.


It seems that also explains why Serhiy, in describing his experiment 
really raw string literals mentioned having to change the tokenizer as 
well as the parser (proving that it isn't impossible to deal with truly 
raw strings).


\" not ending a raw string was certainly a gotcha for me when I started 
using Python (with a background in C and Perl among other languages), 
and it convinced me not to raw strings, that that gotcha was not worth 
the other benefits of raw strings. Serhiy said:
Currently a raw literal cannot end in a single backslash (e.g. in 
r"C:\User\"). Although there are reasons for this. It is an old 
gotcha, and there are many closed issues about it. This question is 
even included in FAQ. 
which indicates that I am not the only one that has been tripped up by 
that over the years.


Trying to look at it from the eyes of a beginning programmer, the whole 
idea of backslash being an escape character is an unnatural artifice. 
I'm unaware (but willing to be educated) of any natural language, when 
using quotations, that has such a  concept. Nested quotations exist, in 
various forms:  use of a different quotation mark for the inner and 
outer quotations, and block quotations (which in English, have increased 
margin on both sides, and have a blank line before and after).


Python actually supports constructs very similar to the natural language 
formats, allowing both  " and ' for quotations and nested quotations, 
and the triple-quoted string with either " or ' is very similar in 
concept to a block quotation. But _all_ the strings forms are burdened 
with surprises for the beginning programmer: escape sequences of one 
sort or another must be learned and understood to avoid surprises when 
using the \ character.


Programming languages certainly need an escape character mechanism to 
deal with characters that cannot easily be typed on a keyboard (such as 
¤ ¶ etc.), or which are visually indistinguishable from other characters 
or character sequences (various widths of white space), or which would 
be disruptive to the flow of code or syntax if represented by the usual 
character (newline, carriage return, formfeed, maybe others). But these 
are programming concepts, not natural language concept.  The basic 
concept of a quoted string should best be borrowed directly from natural 
language, and then enhancements to that made to deal with programming 
concepts.


In Python, as in C, the escape characters are built in the basic string 
syntax, one must learn the quirks of the escaping mechanism in order to 
write


In Perl, " strings include escapes, and ' strings do not. So there is a 
basic string syntax that is similar to natural language, and one that is 
extended to include programming concepts. [N.B. There are lots of 
reasons I switched from Perl to Python, and don't have any desire to go 
back, but I have to admit, that the lack of a truly raw string in Python 
was a disappointment.]


So that, together with the desire for new escape sequences, and the 
creation of a new escape mechanism in the f-string {} (which adds both { 
and } as escape characters by requiring them to be doubled to be treated 
as literal inside an f-string, instead of using \{ and \} as the escapes 
[which would have been possible, due to the addition of the f prefix]), 
and the issue that because every current \-escape is defined to do 
something, is why I suggested elsewhere in this thread 
 
that perhaps the whole irregular string syntax should be rebooted with a 
future import, and it seems it could both be simpler, more regular, and 
more powerful as a result. And by using a future import, there are no 
backward incompatibility issues, and migration can be module by module.


The more I think about this, the more tempting it is to attempt to fork 
Python just to have a better string 

[Python-Dev] Re: Raw string literals and trailing backslash

[Glenn Linderman]

On 8/12/2019 10:21 PM, Serhiy Storchaka wrote:

12.08.19 22:41, Glenn Linderman пише:

On 8/12/2019 12:08 AM, Serhiy Storchaka wrote:
Currently a raw literal cannot end in a single backslash (e.g. in 
r"C:\User\"). Although there are reasons for this. It is an old 
gotcha, and there are many closed issues about it. This question is 
even included in FAQ.


Hmm. I didn't find it documentation, and searching several ways for 
it in a FAQ, I wasn't able to find it either.


https://docs.python.org/3/faq/design.html#why-can-t-raw-strings-r-strings-end-with-a-backslash 



Thanks. After my Google searches failed, I looked at the Python FAQ TOC, 
and the sections that seemed most promising seemed to be "General" and 
"Programming" and "Python on Windows".  I never thought to look under 
"Design and History".  "Programming" actually had a section on strings, 
and it wasn't there... which reduced my enthusiasm for reading the whole 
thing, and since it is in 8 sections, it was cumbersome to do a global 
search in the browser.


It looks like the FAQ is part of the standard documentation, but it 
seems like it would be more useful if there were cross-links between the 
documentation and the related FAQs.




Thanks for your investigation, Serhiy. Point 3 seems like the easiest 
way to convert most regular expressions containing  \" or \'  from  
r"..." form to v"""...""", without disturbing the internal gibberish 
in the regular expression, and without needing significant analysis.


No new prefix is needed, since a single trailing backslash is never a 
problem in regular expression (as it is an illegal RE syntax).


I'd be interested in your comments on my future import idea 
<https://mail.python.org/archives/list/python-dev@python.org/message/XJNS45JG7EUO7EPJG4254HA2T2ASWQ3F/> 
either here or privately. After 30 years of Python, it seems that there 
are quite a few warts in the string syntax, and a fresh start might be 
appropriate, as well as simpler to document, learn, and teach, and 
future import would allow a gradual, opt-in migration. It may be a long 
time, if ever, before the current syntax warts could be removed and the 
future import eliminated, but from the sounds of things, it might also 
be a long time, if ever, before there can be agreement on adding new 
escapes or giving errors for bad ones in the present syntax: making any 
changes without introducing a new prefix is a breaking, incompatible change.





Regarding point 4, if it is a string literal used as a regexp, 
internal triple quotes can be recoded as   "{3}  and  '{3} .


Good point! This is yet one option.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VR34LGEWNJVNKIFNXW7R3CCHFH6USYTT/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SHAEY4H6CKPBEYQ7WU5T2LIBTATRUXKX/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/12/2019 12:11 AM, Serhiy Storchaka wrote:

11.08.19 23:07, Glenn Linderman пише:

On 8/11/2019 1:26 AM, Serhiy Storchaka wrote:

10.08.19 22:10, Glenn Linderman пише:
I wonder how many raw strings actually use the \"  escape 
productively? Maybe that should be deprecated too! ?  I can't think 
of a good and necessary use for it, can anyone?


This is an interesting question. I have performed some experiments. 
15 files in the stdlib (not counting the tokenizer) use \' or \" in 
raw strings. And one test (test_venv) is failed because of using 
them in third-party code. All cases are in regular expressions. It 
is possible to rewrite them, but it is less trivial task than fixing 
invalid escape sequences. So changing this will require much much 
more long deprecation period.


Couldn't they be rewritten using the above idiom? Why would that be 
less trivial?
Or by using triple quotes, so the \" could be written as " ? That 
seems trivial.


Yes, they could. You can use different quote character, triple quotes, 
string literal concatenation. There are many options, and you should 
choose what is applicable in any particular case and what is optimal. 
You need to analyze the whole string literal and code transformation 
usually is more complex than just duplicating a backslash or adding 
the `r` prefix. For example, in many cases `\"` can be replaced with 
`"'"'r"`, but it does not look pretty readable.


No, that is not readable.  But neither does it seem to be valid syntax, 
or else I'm not sure what you are saying. Ah, maybe you were saying that 
a seqence like the '\"' that is already embedded in a raw string can be 
converted to the sequence `"'"'r"` also embedded in the raw string. That 
makes the syntax work, but if that is what you were saying, your 
translation dropped the \ from before the ", since the raw string 
preserves both the \ and the ".


Regarding the readability, I think any use of implicitly concatenated 
strings should have at least two spaces or a newline between them to 
make the implicit concatenation clearer.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VGFQFBKNPNKBJLOPDHQWGCJ6WPK7IDKT/

[Python-Dev] Re: Raw string literals and trailing backslash

[Glenn Linderman]

On 8/12/2019 12:08 AM, Serhiy Storchaka wrote:
Currently a raw literal cannot end in a single backslash (e.g. in 
r"C:\User\"). Although there are reasons for this. It is an old 
gotcha, and there are many closed issues about it. This question is 
even included in FAQ.


Hmm. I didn't find it documentation, and searching several ways for it 
in a FAQ, I wasn't able to find it either.




The most common workarounds are:

    r"C:\User" "\\"

and

    r"C:\User\ "[:-1]

I tried to experiment. It was easy to make the parser allowing a 
trailing backslash character. It was more difficult to change the 
Python implementation in the tokenizer module. But this change breaks 
existing code in more sites than I expected. 14 Python files in the 
stdlib (not counting tokenizer.py) will need to be fixed. In all cases 
it is a regular expression.


Few examples:

1.
    r"([\"\\])"

If only one type of quotes is used in a string, we can just use 
different kind of quotes for creating a string literal and remove 
escaping.


    r'(["\\])'

2.
    r'(\'[^\']*\'|"[^"]*"|...'

If different types o quotes are used in different parts of a string, 
we can use implicit concatenation of string literals created with 
different quotes (in any case a regular expression is long and should 
be split on several lines on semantic boundaries).


    r"('[^']*'|"
    r'"[^"]*"|'
    r'...'

3.
    r"([^.'\"\\#]\b|^)"

You can also use triple quotes if the string contain both type of 
quotes together.


    r"""([^.'"\\#]\b|^)"""

4. In rare cases a multiline raw string literals can contain both 
`'''` and `"""`. In this case you can use implicit concatenation of 
string literals created with different triple quotes.


See https://github.com/python/cpython/pull/15217 .

I do not think we are ready for such breaking change. It will break 
more code than forbidding unrecognized escape sequences, and the 
required fixes are less trivial.


Thanks for your investigation, Serhiy.  Point 3 seems like the easiest 
way to convert most regular expressions containing  \" or \'  from  
r"..." form to v"""...""", without disturbing the internal gibberish in 
the regular expression, and without needing significant analysis.


Regarding point 4, if it is a string literal used as a regexp, internal 
triple quotes can be recoded as   "{3}  and  '{3} .  But whether or not 
it is used as a regexp, I fail to find a syntax that permits the 
creation of a multiline raw string contining both "'''" and '"""', 
without using implicit concatenation. Since implicit concatenation must 
already be in use for that case, converting from  raw string to verbatim 
string is straightforward.




___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BVTHINCVSXGYG5VCIRPP7MAIF2ACWWUZ/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/11/2019 8:40 PM, Eric V. Smith wrote:

On 8/11/2019 4:18 PM, Glenn Linderman wrote:

On 8/11/2019 2:50 AM, Steven D'Aprano wrote:

On Sat, Aug 10, 2019 at 12:10:55PM -0700, Glenn Linderman wrote:


Or invent "really raw" in some spelling, such as rr"c:\directory\"
or e for exact, or x for exact, or "c:\directory\"

And that brings me to the thought that if   \e  wants to become an
escape for escape, that maybe there should be an "extended escape"
prefix... if you want to use more escapes, define ee"string where \\
can only be used as an escape or escaped character, \e means the ASCII
escape character, and \ followed by a character with no escape
definition would be an error."

Please no.

We already have b-strings, r-strings, u-strings, f-strings, br-strings,
rb-strings, fr-strings, rf-strings, each of which comes in four
varieties (single quote, double quote, triple single quote and triple
double quote). Now you're talking about adding rr-strings, v-strings
(Greg suggested that) and ee-strings, presumably some or all of which
will need b*- and *b- or f*- and *f- varieties too.


Don't forget the upper & lower case varieties :)


And all orders!

>>> _all_string_prefixes()
{'', 'b', 'BR', 'bR', 'B', 'rb', 'F', 'RF', 'rB', 'FR', 'Rf', 'Fr', 
'RB', 'f', 'r', 'rf', 'rF', 'R', 'u', 'fR', 'U', 'Br', 'Rb', 'fr', 'br'}

>>> len(_all_string_prefixes())
25

And if you add just 'bv' and 'fv', it's 41:

{'', 'fr', 'Bv', 'BR', 'F', 'rb', 'Fv', 'VB', 'vb', 'vF', 'br', 'FV', 
'vf', 'FR', 'fV', 'bV', 'Br', 'Vb', 'Rb', 'RF', 'bR', 'r', 'R', 'Vf', 
'fv', 'U', 'RB', 'B', 'rB', 'vB', 'Fr', 'rF', 'fR', 'Rf', 'BV', 'VF', 
'bv', 'b', 'u', 'f', 'rf'}


There would be no need for 'uv' (not needed for backward 
compatibility) or 'rv' (can't be both raw and verbatim).


I'm not in any way serious about this. I just want people to realize 
how many wacky combinations there would be. And heaven forbid we ever 
add some combination of 3 characters. If 'rfv' were actually also 
valid, you get to 89:


{'', 'br', 'vb', 'fR', 'F', 'rFV', 'fRv', 'fV', 'rVF', 'Rfv', 'u', 
'vRf', 'fVR', 'rfV', 'Fvr', 'vrf', 'fVr', 'vB', 'Vb', 'Rvf', 'Fv', 
'Fr', 'FVr', 'B', 'rVf', 'FVR', 'vfr', 'VB', 'VrF', 'BR', 'VRf', 
'vfR', 'FR', 'Br', 'RFV', 'Rf', 'fvR', 'f', 'rb', 'VfR', 'VFR', 'fr', 
'vFR', 'VRF', 'frV', 'bR', 'b', 'FrV', 'r', 'R', 'RVF', 'FV', 'rvF', 
'FRV', 'Vrf', 'rvf', 'FRv', 'Frv', 'vF', 'bV', 'VF', 'fv', 'RF', 'RB', 
'rB', 'vRF', 'RFv', 'RVf', 'Rb', 'Vfr', 'vrF', 'rf', 'Bv', 'vf', 'rF', 
'U', 'bv', 'FvR', 'RfV', 'Vf', 'VFr', 'vFr', 'fvr', 'BV', 'rFv', 
'rfv', 'fRV', 'frv', 'RvF'}


If only we could deprecate upper case prefixes!

Eric


Yes. Happily while there is a combinatorial explosion in spellings and 
casings, there is no cognitive overload: each character has an 
independent effect on the interpretation and use of the string, so once 
you understand the 5 existing types (b r u f and plain) you understand 
them all.


Should we add one or two more, it would be with the realization 
(hopefully realized in the documentation also) that v and e would 
effectively be replacements for r and plain, rather than being combined 
with them.


Were I to design a new language with similar string syntax, I think I 
would use plain quotes for verbatim strings only, and have the following 
prefixes, in only a single case:


(no prefix) - verbatim UTF-8 (at this point, I see no reason not to 
require UTF-8 for the encoding of source files)

b - for verbatim bytes
e - allow (only explicitly documented) escapes
f - format strings

Actually, the above could be done as a preprocessor for python, or a 
future import. In other words, what you see is what you get, until you 
add a prefix to add additional processing.  The only combinations that 
seem useful are  eb  and  ef.  I don't know that constraining the order 
of the prefixes would be helpful or not, if it is helpful, I have no 
problem with a canonical ordering being prescribed.


As a future import, one could code modules to either the current 
combinatorial explosion with all its gotchas, special cases, and passing 
of undefined escapes; or one could code to the clean limited cases above.


Another thing that seems awkward about the current strings is that {{ 
and }} become "special escapes".  If it were not for the permissive 
usage of \{ and \} in the current plain string processing, \{ and \} 
could have been used to escape the non-format-expression uses of { and 
}, which would be far more consistent with other escapes.  Perhaps the 
future import could regularize that, also.


A future import would have no backward compatibility issues to disrupt a 
simplified, more regular syntax.


Does anyone know of an existing feature that couldn't be expressed in a 
straightforward manner with only the above capabilities?



The only other thing that I have heard about regarding strings is that 
multi-line strings 

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/11/2019 2:50 AM, Steven D'Aprano wrote:

On Sat, Aug 10, 2019 at 12:10:55PM -0700, Glenn Linderman wrote:


Or invent "really raw" in some spelling, such as rr"c:\directory\"
or e for exact, or x for exact, or "c:\directory\"

And that brings me to the thought that if   \e  wants to become an
escape for escape, that maybe there should be an "extended escape"
prefix... if you want to use more escapes, define   ee"string where \\
can only be used as an escape or escaped character, \e means the ASCII
escape character, and \ followed by a character with no escape
definition would be an error."

Please no.

We already have b-strings, r-strings, u-strings, f-strings, br-strings,
rb-strings, fr-strings, rf-strings, each of which comes in four
varieties (single quote, double quote, triple single quote and triple
double quote). Now you're talking about adding rr-strings, v-strings
(Greg suggested that) and ee-strings, presumably some or all of which
will need b*- and *b- or f*- and *f- varieties too.


Don't forget the upper & lower case varieties :)


If the plan to deprecate unrecognised escapes and then make them an
exception goes ahead, and I expect that it will, in a few more releases
this "extended escape" ee-string will be completely redundent. If \e is
required, we will be able to add it to regular strings as needed,
likewise for any future new escapes we might want. (If any.)
So unrecognized escapes were deprecated in 3.6. And didn't get removed 
in 3.7. And from all indications, aren't going to be removed in 3.8. 
What makes you think the same arguments won't happen again for 3.9?



And if we end up keeping the existing behaviour, oh well, we can always
write \x1B instead. New escapes are a Nice To Have, not a Must Have.

"Really raw" rr'' versus "nearly raw" r'' is a source of confusion just
waiting to happen, when people use the wrong numbers of r's, or are
simply unclear which they should use.
I agree that Greg's v is far better than rr, especially if someone tried 
to write rfr or rbr.

It's not like we have no other options:

 location = r'C:\directory\subdirectory' '\\'

works fine.
But I never thought of that, until Serhiy mentioned it in his reply, so 
there are probably lots of other stupid people that didn't think of it 
either. It's not like it is even suggested in the documentation as a way 
to work around the non-rawness of raw strings. And it still requires 
doubling one of the \, so it is more consistent and understandable to 
just double them all.



  So does this:

 location = 'directory/subdirectory/'.replace('/', os.sep)


This is a far greater run-time cost with the need to scan the string. 
Granted the total cost isn't huge, unless it is done repeatedly.



Even better, instead of hard-coding our paths in the source code, we can
read them from a config file or database.
Yep, I do that sometimes. But hard-coded paths make good defaults in 
many circumstances.



It is unfortunate that Windows is so tricky with backslashes and
forwards slashes, and that it clashes with the escape character, but I'm
sure that other languages which use \ for escaping haven't proliferated
a four or more kinds of strings with different escaping rules in
response.


I agree with this. But Bill didn't consult Guido about the matter.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BPW6VYVKANWICN34TIOA6BVJYXX4MK3D/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/11/2019 1:26 AM, Serhiy Storchaka wrote:

10.08.19 22:10, Glenn Linderman пише:
As pointed out elsewhere, Raw strings have limitations, paths ending 
in \ cannot be represented, and such do exist in various situations, 
not all of which can be easily avoided... except by the "extra 
character contortion" of "C:\directory\ "[:-1]  (does someone know a 
better way?)


Other common idiom is

    r"C:\directory" "\\"


I suppose that concatenation happens at compile time; less sure about 
[:-1], I would guess not. Thanks for this.


I wonder how many raw strings actually use the \"  escape 
productively? Maybe that should be deprecated too! ?  I can't think 
of a good and necessary use for it, can anyone?


This is an interesting question. I have performed some experiments. 15 
files in the stdlib (not counting the tokenizer) use \' or \" in raw 
strings. And one test (test_venv) is failed because of using them in 
third-party code. All cases are in regular expressions. It is possible 
to rewrite them, but it is less trivial task than fixing invalid 
escape sequences. So changing this will require much much more long 
deprecation period.


Couldn't they be rewritten using the above idiom? Why would that be less 
trivial?
Or by using triple quotes, so the \" could be written as " ? That seems 
trivial.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/46TOPB5ZY24OXXBGSLUXOQOJOASGBVTL/

[Python-Dev] Re: An f-string issue [Was: Re: Re: What to do about invalid escape sequences]

[Glenn Linderman]

On 8/10/2019 5:32 PM, Greg Ewing wrote:

Glenn Linderman wrote:
If that were true, the \n in the above example would already be a 
newline character, and the parsing of the format expression would not 
see the backslash. And if it were true, that would actually be far

more useful for this situation.


But then it would fail for a different reason -- the same reason that
this is a syntax error:

   'hello
   world'


Would it really?  Or would it, because it has already been lexed and 
parsed as string content by then, simply be treated as a new line that 
is part of the string? just like "hello\nworld" is treated after it is 
lexed and parsed?


Of course, if it is passed back through the parser again, you would be 
correct. I don't know the internals that apply here.


Anyway, Eric supplied the real reasons for the limitation, but it does 
seem like if it would be passed back through the "real" parser, that the 
real parser would have no problem handling the ord('\n') part of


f"newline: {ord('\n')}"

if it weren't prohibited by prechecking for \ and making it illegal. But 
there is also presently a custom parser involved, so whether the \ check 
is in there or in a preprocessing step before the parser, I don't know.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/THSM262IIVNYRF2DDSXNPYPSLSK5W3GY/

[Python-Dev] An f-string issue [Was: Re: Re: What to do about invalid escape sequences]

[Glenn Linderman]

Because of the "invalid escape sequence" and "raw string" discussion, 
when looking at the documentation, I also noticed the following 
description for f-strings:


Escape sequences are decoded like in ordinary string literals (except 
when a literal is also marked as a raw string). After decoding, the 
grammar for the contents of the string is:

followed by lots of stuff, followed by

Backslashes are not allowed in format expressions and will raise an error:
f"newline: {ord('\n')}"   # raises SyntaxError


What I don't understand is how, if f-strings are processed AS DESCRIBED, 
how the \n is ever seen by the format expression.


The description is that they are first decoded like ordinary strings, 
and then parsed for the internal grammar containing {} expressions to be 
expanded.  If that were true, the \n in the above example would already 
be a newline character, and the parsing of the format expression would 
not see the backslash. And if it were true, that would actually be far 
more useful for this situation.


So given that it is not true, why not? And why go to the extra work of 
prohibiting \ in the format expressions?


The PEP 498, of course, has an apparently more accurate description, 
that the {} parsing actually happens before the escape processing. 
Perhaps this avoids making multiple passes over the string to do the 
work, as the literal pieces and format expression pieces have to be 
separate in the generated code, but that is just my speculation: I'd 
like to know the real reason.


Should the documentation be fixed to make the description more accurate? 
If so, I'd be glad to open an issue.


The PEP further contains the inaccurate statement:

Like all raw strings in Python, no escape processing is done for raw 
f-strings:


not mentioning the actual escape processing that is done for raw 
strings, regarding \" and \'.






___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OH3APTWUWPDC376FBRKNEXBKQYPP6LXY/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/10/2019 3:36 PM, Greg Ewing wrote:

Glenn Linderman wrote:


I wonder how many raw strings actually use the \"  escape 
productively? Maybe that should be deprecated too! ?  I can't think 
of a good and necessary use for it, can anyone?


Quite rare, I expect, but it's bound to break someone's code.
It might be better to introduce a new string prefix, e.g.
'v' for 'verbatim':

   v"C:\Users\Fred\"

Which is why I suggested  rr"C:\directory\", but allowed as how there 
might be better spellings I like your  v for verbatim !
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GTOVRKM7Q4VU67KYDQF6ICU7HAJDSBRX/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/10/2019 12:19 PM, Guido van Rossum wrote:

Regular expressions.


I assume that is in response to the "good use for \" escape" question?

But can't you just surround them with ' instead of " ?  Or  ''' ?



On Sat, Aug 10, 2019 at 12:12 Glenn Linderman <mailto:v%2bpyt...@g.nevcal.com>> wrote:


On 8/10/2019 11:16 AM, Terry Reedy wrote:

On 8/10/2019 4:33 AM, Paul Moore wrote:


(Side issue)


This deserves its own thread.


As a Windows developer, who has seen far too many cases where
use of
slashes in filenames implies a Unix-based developer not thinking
sufficiently about Windows compatibility, or where it leads to
people
hard coding '/' rather than using os.sep (or better, pathlib), I
strongly object to this characterisation. Rather, I would simply
say
"to make Windows users more aware of the clash in usage between
backslashes in filenames and backslashes as string escapes".

There are *many* valid ways to write Windows pathnames in your
code:

1. Raw strings


As pointed out elsewhere, Raw strings have limitations, paths
ending in \ cannot be represented, and such do exist in various
situations, not all of which can be easily avoided... except by
the "extra character contortion" of "C:\directory\ "[:-1]  (does
someone know a better way?)

It would be useful to make a "really raw" string that doesn't
treat \ special in any way. With 4 different quoting possibilities
( ' " ''' """ ) there isn't really a reason to treat \ special at
the end of a raw string, except for backward compatibility.

I wonder how many raw strings actually use the \"  escape
productively? Maybe that should be deprecated too! ?  I can't
think of a good and necessary use for it, can anyone?

Or invent "really raw" in some spelling, such as rr"c:\directory\"
or e for exact, or x for exact, or "c:\directory\"

And that brings me to the thought that if   \e  wants to become an
escape for escape, that maybe there should be an "extended escape"
prefix... if you want to use more escapes, define   ee"string
where \\ can only be used as an escape or escaped character, \e
means the ASCII escape character, and \ followed by a character
with no escape definition would be an error."

Of course "extended escape" could be spelled lots of different
ways too, but not the same way as "really raw" :)


2. Doubling the backslashes
3. Using pathlib (possibly with slash as a directory separator,
where
it's explicitly noted as a portable option)
4. Using slashes

IMO, using slashes is the *worst* of these. But this latter is a
matter of opinion - I've no objection to others believing
differently,
but I *do* object to slashes being presented as the only option, or
the recommended option without qualification.


Perhaps Python Setup and Usage, 3. Using Python on Windows,
should have a section of file paths, at most x.y.z, so visible in
the TOC listed by https://docs.python.org/3/using/index.html



___
Python-Dev mailing list -- python-dev@python.org
<mailto:python-dev@python.org>
To unsubscribe send an email to python-dev-le...@python.org
<mailto:python-dev-le...@python.org>
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at

https://mail.python.org/archives/list/python-dev@python.org/message/5MZAXJJYKNMQAS63QW4HS2TUPMQH7LSL/

--
--Guido (mobile)


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BZDAXLX2IQTIUT2W47SFI2CJTZSPXY2V/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/10/2019 11:16 AM, Terry Reedy wrote:

On 8/10/2019 4:33 AM, Paul Moore wrote:


(Side issue)


This deserves its own thread.


As a Windows developer, who has seen far too many cases where use of
slashes in filenames implies a Unix-based developer not thinking
sufficiently about Windows compatibility, or where it leads to people
hard coding '/' rather than using os.sep (or better, pathlib), I
strongly object to this characterisation. Rather, I would simply say
"to make Windows users more aware of the clash in usage between
backslashes in filenames and backslashes as string escapes".

There are *many* valid ways to write Windows pathnames in your code:

1. Raw strings


As pointed out elsewhere, Raw strings have limitations, paths ending in 
\ cannot be represented, and such do exist in various situations, not 
all of which can be easily avoided... except by the "extra character 
contortion" of   "C:\directory\ "[:-1]  (does someone know a better way?)


It would be useful to make a "really raw" string that doesn't treat \ 
special in any way. With 4 different quoting possibilities ( ' " ''' """ 
) there isn't really a reason to treat \ special at the end of a raw 
string, except for backward compatibility.


I wonder how many raw strings actually use the \"  escape productively? 
Maybe that should be deprecated too! ?  I can't think of a good and 
necessary use for it, can anyone?


Or invent "really raw" in some spelling, such as rr"c:\directory\"
or e for exact, or x for exact, or here>"c:\directory\"


And that brings me to the thought that if   \e  wants to become an 
escape for escape, that maybe there should be an "extended escape" 
prefix... if you want to use more escapes, define   ee"string where \\ 
can only be used as an escape or escaped character, \e means the ASCII 
escape character, and \ followed by a character with no escape 
definition would be an error."


Of course "extended escape" could be spelled lots of different ways too, 
but not the same way as "really raw" :)



2. Doubling the backslashes
3. Using pathlib (possibly with slash as a directory separator, where
it's explicitly noted as a portable option)
4. Using slashes

IMO, using slashes is the *worst* of these. But this latter is a
matter of opinion - I've no objection to others believing differently,
but I *do* object to slashes being presented as the only option, or
the recommended option without qualification.


Perhaps Python Setup and Usage, 3. Using Python on Windows, should 
have a section of file paths, at most x.y.z, so visible in the TOC 
listed by https://docs.python.org/3/using/index.html




___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5MZAXJJYKNMQAS63QW4HS2TUPMQH7LSL/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/10/2019 7:03 AM, Paul Moore wrote:

On Sat, 10 Aug 2019 at 12:06, Chris Angelico  wrote:

On Sat, Aug 10, 2019 at 6:39 PM Paul Moore  wrote:

There are *many* valid ways to write Windows pathnames in your code:

1. Raw strings
2. Doubling the backslashes
3. Using pathlib (possibly with slash as a directory separator, where
it's explicitly noted as a portable option)
4. Using slashes

IMO, using slashes is the *worst* of these. But this latter is a
matter of opinion - I've no objection to others believing differently,
but I *do* object to slashes being presented as the only option, or
the recommended option without qualification.

Please expand on why this is the worst?

I did say it was a matter of opinion, so I'm not going to respond if
people say that any of the following is "wrong", but since you asked:

1. Backslash is the native separator, whereas slash is not (see eryk
sun's post for *way* more detail).
2. People who routinely use slash have a tendency to forget to use
os.sep rather than a literal slash in places where it *does* matter.
3. Using slash, in my experience, ends up with paths with "mixed"
separators (os.path.join("C:/work/apps", "foo") ->
'C:/work/apps\\foo') which are messy to deal with, and ugly for the
user.
4. If a path with slashes is displayed directly to the user without
normalisation, it looks incorrect and can confuse users who are only
used to "native" Windows programs.

Etc.
Not to mention the problem of passing paths with / to other windows 
programs via system or subprocess.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/A7MBGUBTRNLZ5UWCMS4NHYAFGQC6MNQJ/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/9/2019 3:56 PM, Steven D'Aprano wrote:

I'm not trying to be confrontational, I'm trying to understand your
use-case(s) and see if it would be broken by the planned change to
string escapes.


Yeah, that's fine. Sometimes it is hard to communicate via email (versus 
saying a lot).



On Fri, Aug 09, 2019 at 03:18:29PM -0700, Glenn Linderman wrote:

On 8/9/2019 2:53 PM, Steven D'Aprano wrote:

On Fri, Aug 09, 2019 at 01:12:59PM -0700, Glenn Linderman wrote:


The reason I never use raw strings is in the documentation, it is
because \ still has a special meaning, and the first several times I
felt the need for raw strings, it was for directory names that wanted to
end with \ and couldn't.

Can you elaborate? I find it unlikely that I would ever want a docstring

I didn't mention docstring.  I just wanted a string with a path name
ending in \.

You said you never used raw strings in the documentation. I read that as
doc strings. What sort of documentation are you writing that isn't a doc
string but is inside your .py files where the difference between raw and
regular strings is meaningful?


No, what I said was that the reason is in the documentation. The reason 
that I don't use raw strings is in the Python documentation. I don't 
claim to use raw strings for documentation I write. The reason is 
because \" to end the string doesn't work, and the first good-sounding 
justification for using raw strings that I stumbled across was to avoid 
"c:\\directory\\" in favor of r"c:\directory\"  but that doesn't work, 
and neither do r"c:\directory\\". Since then, I have not found any other 
compelling need for raw strings that overcome that deficiency... the 
benefit of raw strings is that you don't have to double the \\. But the 
benefit is contradicted by not being able to use one at the end of 
sting. If you can't use it at the end of the string, the utility of not 
doubling them in the middle of the string is just too confusing to make 
it worth figuring out the workarounds when you have a string full of \ 
that happens to end in \. Just easier to remember the "always double \" 
rule, than to remember the extra "but if your string containing \ 
doesn't have one at the end you can get away with using a raw string and 
not doubling the \.



Windows users are used to seeing backslashes in paths, I don't care to
be the one to explain why my program uses / and all the rest use \.

If you don't use raw strings for paths, you get to explain why your
program uses \\ and all the rest use \ *wink*
Wrong. Users don't look at the source code. They look at the output. I 
also don't want to have to write code to convert /-laden paths to 
\-laden paths when I display them to the user.




If they're Windows end users, they won't be reading your source code and
will never know how you represent hard-coded paths in the source code.


They will if I display the path as a default value for an argument, or 
show them the path for other reasons, or if the path shows up in an 
exception message.




If they're Windows developers, they ought to be aware that the Windows
file system API allows / anywhere you can use \ and it is the
common convention in Python to use forward slashes.


This, we can agree on.


I'm also curious why the string needs to *end* with a backslash. Both of
these are the same path:

 C:\foo\bar\baz\
 C:\foo\bar\baz


Sure. But only one of them can be used successfully with   + filename  
(for example).


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OYRSO4WHUFA7Q34HJTBIMQL337JTA5RX/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/9/2019 4:08 PM, MRAB wrote:

On 2019-08-09 23:56, Steven D'Aprano wrote:

I'm not trying to be confrontational, I'm trying to understand your
use-case(s) and see if it would be broken by the planned change to
string escapes.


On Fri, Aug 09, 2019 at 03:18:29PM -0700, Glenn Linderman wrote:

On 8/9/2019 2:53 PM, Steven D'Aprano wrote:
>On Fri, Aug 09, 2019 at 01:12:59PM -0700, Glenn Linderman wrote:
>
>>The reason I never use raw strings is in the documentation, it is
>>because \ still has a special meaning, and the first several times I
>>felt the need for raw strings, it was for directory names that 
wanted to

>>end with \ and couldn't.
>Can you elaborate? I find it unlikely that I would ever want a 
docstring


I didn't mention docstring.  I just wanted a string with a path name 
ending in \.


You said you never used raw strings in the documentation. I read that as
doc strings. What sort of documentation are you writing that isn't a doc
string but is inside your .py files where the difference between raw and
regular strings is meaningful?


Windows users are used to seeing backslashes in paths, I don't care 
to be the one to explain why my program uses / and all the rest use \.


If you don't use raw strings for paths, you get to explain why your
program uses \\ and all the rest use \ *wink*

If they're Windows end users, they won't be reading your source code and
will never know how you represent hard-coded paths in the source code.

If they're Windows developers, they ought to be aware that the Windows
file system API allows / anywhere you can use \ and it is the
common convention in Python to use forward slashes.

I'm also curious why the string needs to *end* with a backslash. Both of
these are the same path:

 C:\foo\bar\baz\
 C:\foo\bar\baz



The only time it's required is for the root directory of a drive:

C:\


That's not the only time it's required, but it is a case that is far 
harder to specify in other ways.  It's required any time you  want to 
say   + filename without writing + "\\" + filename, or os.path.join( 
'C:\\", filename )
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LXM72OGMFTNJP3NQPITJWWGB6ITNRBH4/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/9/2019 4:07 PM, Gregory P. Smith wrote:



On Fri, Aug 9, 2019 at 11:37 AM Eric V. Smith > wrote:


On 8/9/2019 2:28 PM, Jonathan Goble wrote:
> On Fri, Aug 9, 2019 at 12:34 PM Nick Coghlan mailto:ncogh...@gmail.com>> wrote:
>> I find the "Our deprecation warnings were even less visible than
>> normal" argument for extending the deprecation period compelling.
> Outsider's 2 cents from reading this discussion (with no personal
> experience with this warning):
>
> I am perplexed at the opinion, seemingly espoused by multiple people
> in this thread, that because a major part of the problem is that the
> warnings were not visible enough, somehow the proposed solution is
> making them not visible enough again? It's too late, in my
> understanding, in the 3.8 cycle to add a new feature like a
change to
> how these warnings are produced (it seems a significant change
to the
> .pyc structure is needed to emit them at runtime), so this supposed
> "solution" is nothing but kicking the can down the road. When 3.9
> rolls around, public exposure to the problem of invalid escape
> sequences will still be approximately what it is now (because if
> nobody saw the warnings in 3.7, they certainly won't see them in 3.8
> with this "fix"), so you'll end up with the same complaints about
> SyntaxWarning that started this discussion, end up back on
> DeprecationWarning for 3.9 (hopefully with support for emitting them
> at runtime instead of just compile-time), then have to wait until
> 3.10/4.0 for SyntaxWarning and eventually the next version to
actually
> make them errors.

Yes, I think that's the idea: Deprecation warning in 3.9, but more
visible that what 3.7 has. That is, not just at compile time but
at run
time. What's required to make that happen is an open question.


i've lost track of who suggested what in this thread, but yes, that 
concept has been rolling over in my mind as a potentially good idea 
after someone suggested it.  Compile time warnings should turn into 
bytecode for a warnings.warn call in the generated pyc.  I haven't 
spent time trying to reason if that actually addresses the real issues 
we're having moving forward with a syntax warning change though. A 
reasonable feature to ask for as a feature in 3.9 or later perhaps.


The documentation actually claims it was deprecated in version 3.6. So 
it has already been 2 releases worth of deprecation, visible warning or not.


Ship it.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YYSC5CDJWOF24AUWC4IHJG45COHOTHW3/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/9/2019 2:53 PM, Steven D'Aprano wrote:

On Fri, Aug 09, 2019 at 01:12:59PM -0700, Glenn Linderman wrote:


The reason I never use raw strings is in the documentation, it is
because \ still has a special meaning, and the first several times I
felt the need for raw strings, it was for directory names that wanted to
end with \ and couldn't.

Can you elaborate? I find it unlikely that I would ever want a docstring


I didn't mention docstring.  I just wanted a string with a path name 
ending in \.



that ends with a backslash:

 def func():
 r"""Documentation goes here...
 more documentation...
 ending with a Windows path that needs a trailing backslash
 like this C:\directory\"""

That seems horribly contrived. Why use backslashes in the path when the
strong recommendation is to use forward slashes?


Windows users are used to seeing backslashes in paths, I don't care to 
be the one to explain why my program uses / and all the rest use \.



And why not solve the problem by simply moving the closing quotes to the
next line, as PEP 8 recommends?

 r"""Documentation ...
 C:\directory\
 """


This isn't my problem, I wasn't using docstrings, and including a 
newline in a path name doesn't work.  I suppose one could "solve" the 
problem by using


"c:\directory\ "[ :-1]

but that is just as annoying as

"c:\\directory\\"

and back when I discovered the problem, I was still learning Python, and 
didn't think of the above solution either.





[...]

Even in a raw literal, quotes can be escaped with a backslash

Indeed, they're not so much "raw" strings as only slightly blanched
strings.




___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EHE2CNRDGS6AF6GYO4DX7UNNE24JH6CG/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/9/2019 9:08 AM, Nick Coghlan wrote:

On Sat, 10 Aug 2019 at 01:44, Guido van Rossum  wrote:

This discussion looks like there's no end in sight. Maybe the Steering Council 
should take a vote?

I find the "Our deprecation warnings were even less visible than
normal" argument for extending the deprecation period compelling.

I also think the UX of the warning itself could be reviewed to provide
a more explicit nudge towards using raw strings when folks want to
allow arbitrary embedded backslashes. Consider:

 SyntaxWarning: invalid escape sequence \,

vs something like:

 SyntaxWarning: invalid escape sequence \, (Note: adding the raw
string literal prefix, r, will accept all non-trailing backslashes)

After all, the habit we're trying to encourage is "If I want to
include backslashes without escaping them all, I should use a raw
string", not "I should memorize the entire set of valid escape
sequences" or even "I should always escape backslashes".

Cheers,
Nick.

The reason I never use raw strings is in the documentation, it is 
because \ still has a special meaning, and the first several times I 
felt the need for raw strings, it was for directory names that wanted to 
end with \ and couldn't. Quoted below. Also relevant to the discussion 
is the "benefit" of leaving the backslash in the result of an illegal 
escape, which no one has mentioned in this huge thread.


Unlike Standard C, all unrecognized escape sequences are left in the 
string unchanged, i.e., /the backslash is left in the result/. (This 
behavior is useful when debugging: if an escape sequence is mistyped, 
the resulting output is more easily recognized as broken.) It is also 
important to note that the escape sequences only recognized in string 
literals fall into the category of unrecognized escapes for bytes 
literals.


Changed in version 3.6: Unrecognized escape sequences produce a
DeprecationWarning. In some future version of Python they will be
a SyntaxError.

Even in a raw literal, quotes can be escaped with a backslash, but the 
backslash remains in the result; for example, |r"\""| is a valid 
string literal consisting of two characters: a backslash and a double 
quote; |r"\"| is not a valid string literal (even a raw string cannot 
end in an odd number of backslashes). Specifically, /a raw literal 
cannot end in a single backslash/ (since the backslash would escape 
the following quote character). Note also that a single backslash 
followed by a newline is interpreted as those two characters as part 
of the literal, /not/ as a line continuation.




___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HKLW5VBZK46TOP6WURFH767YCHRFOYNN/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/7/2019 6:13 PM, raymond.hettin...@gmail.com wrote:

This isn't about me.  As a heavy user of the 3.8 beta, I'm just the canary in 
the coal mine.
Are you, with an understanding of the issue, submitting bug reports on 
the issues you find, thus helping to alleviate the problem, and educate 
the package maintainers?


Or are you just carping here?

I'll apologize in advance for using the word "carping" if the answer to 
my first question is yes. :)


Glenn
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/N26WJ2BCYT7CPFRHZGLQKILDCCKDTV5N/

[Python-Dev] Re: What to do about invalid escape sequences

[Glenn Linderman]

On 8/6/2019 5:57 PM, Gregory P. Smith wrote:
People distribute code via pypi.  if we reject uploads of packages 
with these problems and link to fixers (modernize can be taught what 
to do), we prevent them from spreading further.  A few years after 
doing that, we can revisit how much pain and for whom making this a 
SyntaxWarning or even SyntaxError would actually be.


it isn't useful to tell /users/ of packages to spend time figuring out 
who to complain to that some packages code that they somehow depend on 
(often transitively) is not modern enough.




People also distribute code other ways. A few years after cleaning up 
pypi, you still wouldn't know how much pain and for whom making this a 
SyntaxWarning or SyntaxError would actually be.


Just do it and get it over with.

Glenn
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UZLAFCN64OQ3IQ6GPIVFIELSK43DXYXE/

[Python-Dev] Re: strip behavior provides inconsistent results with certain strings

[Glenn Linderman]

On 7/2/2019 12:56 PM, Terry Reedy wrote:

On 7/2/2019 12:16 AM, Glenn Linderman wrote:

On 7/1/2019 8:28 PM, Terry Reedy wrote:


I did not read much of the thread, but the proposal is to wrap a 
near-trivial expression (2 operations) or replace a current method 
call with a method call than is more or less the same len as what it 
replaces.


>>> 'prefix_more_suffix'[len('prefix'):]  # if know s begins with p
'_more_suffix'

>>> 'prefix_more_suffix'.replace('prefix', '')
'_more_suffix'

>>> 'prefix_more_suffix'.strip_pre('prefix')  # proposed


Your example isn't equivalent: .replace would replace multiple 
instances of prefix, not necessarily at the beginning. The .strip_pre 
(however spelled) would strip one instance, only if it is at the 
beginning.


I left out the slightly longer regex which does exactly that.

>>> re.sub('^prefix', '', 'prefix_more_suffix')
'_more_suffix'
>>> re.sub('^prefix', '', 'Xprefix_more_suffix')
'Xprefix_more_suffix'

or with minor variation, deletes only suffixes.

>>> re.sub('suffix$', '', 'prefix_more_suffix')
'prefix_more_'

Are these special cases, which can be easily generalized to match more 
than one string, and to replace rather than delete, special enough to 
merit a new method each.  This remains to be demonstrated.


The more I thought about this, the more I think a more functional 
goal is a variation of replace that works only on one end or the 
other, rather than a variation of strip. I outlined that in a 
different branch in this thread.


To me, a new parameter to str.replace would have a lower bar to 
acceptance.


two extra boolean parameters has been suggested; alternatives would be a 
single string or flag parameter. replace already accepts a count 
parameter for maximum number of replacements from the start. One could 
extend that to pass a negative count parameter to mean how many from the 
end, which is similar to negative indexing for slices, which wouldn't 
even need an extra parameter, but also wouldn't restrict it to the exact 
front or end.  Together with one extra boolean to declare the 
replacement must be at the very front (for positive count) or very end 
(for negative count), would also allow removal of repeated prefixes or 
suffixes, although I'm not sure that would be a common case:


text = 'Hello Hello Hello Is this Fred?'
text.replace('Hello ', '', 2, True )

would leave one Hello.

Of course it wouldn't have that effect with a non-empty replacement:

text.replace('Hello ', 'Greetings', 2, True )

would leave two Hello.


The other documentation issue I noticed is that the 2nd and 3rd 
parameters to startswith and endswith are not fully documented. 
Typically startswith and endswitch are in the logic prior to the 
strip/replace operation, and typically only use the first parameter, 
but looking at their documentation as part of this discussion, I 
found it lacking.


File an issue?  Preferably with a suggested change.

To suggest a change, I'd first have run lots of experiments to figure 
out what they do... and what the edge conditions are... or, I'd have to 
read the implementation to figure it out. I've never used anything but 
the first parameter and reading the documentation for the other two 
raises more questions than it answers.  But I'll open the issue, maybe 
someone can quickly answer the questions I raise, but the terseness of 
the description of the parameters leaves a lot of room for speculation.


https://bugs.python.org/issue37490
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JNSATSGJG37LLDWIO5AKBU5DSTBKUHXF/

[Python-Dev] Re: strip behavior provides inconsistent results with certain strings

[Glenn Linderman]

On 7/1/2019 8:28 PM, Terry Reedy wrote:

On 7/1/2019 1:57 PM, Chris Barker via Python-Dev wrote:

This was quite extensively discussed on python-ideas recently:

https://mail.python.org/archives/list/python-id...@python.org/thread/RJARZSUKCXRJIP42Z2YBBAEN5XA7KEC3/#WIRID57ESUFUAQQQ6ZUY2RK5PKQQYSJ3 



The claim of 'inconsistent results' is based on not reading the doc.

(I'm finding it hard to find a good thread view in the new interface 
-- but that will get you started)


My memory of that thread is that there was a lot of bike shedding, 
and quite a lot of resistance to adding a couple new methods, which I 
personally never understood (Not why we don't want to add methods 
willy-nilly, but why there was this much resistance to what seems 
like an low-disruption, low maintenance, and helpful addition)


I did not read much of the thread, but the proposal is to wrap a 
near-trivial expression (2 operations) or replace a current method 
call with a method call than is more or less the same len as what it 
replaces.


>>> 'prefix_more_suffix'[len('prefix'):]  # if know s begins with p
'_more_suffix'

>>> 'prefix_more_suffix'.replace('prefix', '')
'_more_suffix'

>>> 'prefix_more_suffix'.strip_pre('prefix')  # proposed


Your example isn't equivalent: .replace would replace multiple instances 
of prefix, not necessarily at the beginning. The .strip_pre (however 
spelled) would strip one instance, only if it is at the beginning.


The more I thought about this, the more I think a more functional goal 
is a variation of replace that works only on one end or the other, 
rather than a variation of strip. I outlined that in a different branch 
in this thread.


The other documentation issue I noticed is that the 2nd and 3rd 
parameters to startswith and endswith are not fully documented. 
Typically startswith and endswitch are in the logic prior to the 
strip/replace operation, and typically only use the first parameter, but 
looking at their documentation as part of this discussion, I found it 
lacking.


A method could raise instead of returning the string as-is if the 
prefix is not really a prefix.  How often is this needed?  The most 
common end deletions are whitespace, which the current .strip handles 
correctly.


raising wouldn't be helpful in most of the situations where I use this 
logic... it would require a more complex flow control than the if 
startswith path in the current situation.


Yes, I use strip more frequently, but if startswith: chop_prefix 
operation (and the other end too) happens an awful lot.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BKRBB3HESKT7N7QL66R7UVUYXOBUAQQO/

[Python-Dev] Re: strip behavior provides inconsistent results with certain strings

[Glenn Linderman]

On 6/28/2019 10:10 AM, MRAB wrote:

On 2019-06-28 03:40, Glenn Linderman wrote:

On 6/27/2019 3:09 PM, Brett Cannon wrote:
My guess is that without Guido to just ask this will have to go to a 
PEP as it changes a built-in.

How does adding two new methods change a built-in?

Now if an extra parameter were added to modify lstrip, rstrip, and 
strip to make them do something different, yes.


But adding new methods doesn't change anything, unless someone is 
checking for their existence.


My preferred color is  pstrip and sstrip (prefix and suffix strip) 
since lstrip and rstrip mean left and right.


And maybe there should be a psstrip, that takes two parameters, the 
prefix and the suffix to strip.


Such functions would certainly reduce code in a lot of places where I do

if  string.startswith('foo'):
 string = string[ 3: ];

as well as making it more robust, because the string and its length 
have to stay synchronized when changes are made.


lstrip and rstrip remove treat the argument as a set of characters and 
can remove multiple characters, so having additional methods pstrip 
and sstrip with their very similar names but different behaviours 
could be confusing.


I'd prefer to stick with lXXX and rXXX. ldrop and rdrop, maybe?


I was thinking of the change from stripping sets of characters versus 
stripping specific sequences of characters, all being strip operations.


But when you point out the confusion and suggest the change of names, I 
got to thinking about names, and really, the   pstrip/ldrop or 
sstring/rdrop  operations are more like  "replace" than "strip"... there 
has to be an exact match.


So maybe a better solution to the functionality needed would be lreplace 
and rreplace.  On the other hand,  the existing   .replace( prefix, '', 
1 )  almost implements lreplace! It doesn't require it to be a prefix 
though :(  And there is no option for doing the last N replacements, nor 
requiring a single one to be a suffix.  The last N replacements could be 
done by using a negative N, but it would seem an additional parameter or 
new name would be required to do prefix- and suffix-restricted operations.


So my example above could be written as either:

string.lreplace('foo')
string.lreplace('foo', '')

But calling the operations lreplace and rreplace, including the option 
of putting in replacement strings, would actually be quite handy.  Think 
of this:


filename.rreplace('.html', '.css', True )

where the first parameter is the suffix to remove, the second parameter 
is a suffix to replace it with, and the third parameter says to add the 
replacement even if the first parameter is not found/removed.  That's 
another operation I find frequently in batch file processing, that 
otherwise takes several lines.


As Ned noted in another branch of this discussion, it would be good to 
have an update for the 2.7 documentation.  And if new functions are 
added, by any name, it would be good to have the 3.x documentation 
clearly reference in both directions between lstrip and lreplace, and 
between rstrip and rreplace (or whatever names).


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GRPEPJIPGOWEFHZLWWRT3DKGX3XWDSAM/

[Python-Dev] Re: strip behavior provides inconsistent results with certain strings

[Glenn Linderman]

On 6/27/2019 3:09 PM, Brett Cannon wrote:

My guess is that without Guido to just ask this will have to go to a PEP as it 
changes a built-in.

How does adding two new methods change a built-in?

Now if an extra parameter were added to modify lstrip, rstrip, and strip 
to make them do something different, yes.


But adding new methods doesn't change anything, unless someone is 
checking for their existence.


My preferred color is  pstrip and sstrip (prefix and suffix strip) since 
lstrip and rstrip mean left and right.


And maybe there should be a psstrip, that takes two parameters, the 
prefix and the suffix to strip.


Such functions would certainly reduce code in a lot of places where I do

if  string.startswith('foo'):
    string = string[ 3: ];

as well as making it more robust, because the string and its length have 
to stay synchronized when changes are made.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GNQGK7QFHQPCP3PIHOPY3RVNE5AMJ247/

Re: [Python-Dev] [PEP 558] thinking through locals() semantics

[Glenn Linderman]

On 5/27/2019 7:28 PM, Steven D'Aprano wrote:

On the other hand, locals() currently returns a dict everywhere. It
might be surprising for it to start returning a proxy object inside
functions instead of a dict.
I thought the proxy object sounded more useful... how different is it in 
use from a dict? "proxy" sounds like it should quack like a dict, as a 
general term, but maybe a more specific "proxy" is meant here, that 
doesn't quite quack like a dict?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: update 1

[Glenn Linderman]

On 5/24/2019 9:09 PM, Random832 wrote:

On Thu, May 23, 2019, at 15:27, Steve Holden wrote:

Besides which, it would be lovely to have a major release that didn't
involve any pain at all for the majority of users!

Our erstwhile BDFL always eschewed two-digit version identifiers- due
to the possibility for confusion about collating sequence, I beleive..
We should honour his preferences by going from 3.9 to 4.0.

Is 4.0 still gonna install as "python3", pip3, etc, executable as "py -3" on 
windows? I can see that being a pain point even if nothing else is.


Maybe by then it can go back to just being python, as 2.x will be past 
end-of-life.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Glenn Linderman]

On 5/22/2019 4:09 AM, Christian Heimes wrote:

On 22/05/2019 01.11, Glenn Linderman wrote:

On 5/21/2019 2:00 PM, Nathaniel Smith wrote:

On Tue, May 21, 2019 at 10:43 AM Glenn Linderman  wrote:

After maintaining my own version of http.server to fix or workaround some of 
its deficiencies for some years, I discovered bottle.py. It has far more 
capability, is far better documented, and is just as quick to deploy. While I 
haven't yet converted all past projects to use bottle.py, it will likely happen 
in time, unless something even simpler to use is discovered, although I can 
hardly imagine that happening.

bottle.py uses http.server for its local development mode (the one you
see in their quickstart example at the top of their README). Same with
flask, django, and probably a bunch of other frameworks. It's *very*
widely used.

-n


The source for bottle.py version 0.13-dev has an import for http.client, but not http.server. I 
hadn't tracked down every indirect dependency in the bottle.py source code, but it seems that if 
one uses the "default server" for bottle, that it is "wsgiref", imported from 
wsgiref.simple_server, and that in turn does import BaseHTTPRequestHandler and HTTPServer from 
http.server.

It is the higher-level code in http.server that has significant deficiencies that have caused me 
problems over the years... a "SimpleHTTPRequestHandler" that is so simple it doesn't do 
POST, PUT or PASTE, a "CGIHTTPRequestHandler" that only implements part of the CGI 
protocol, only CGI support in POST, no support for PUT or PASTE, and no support for https, and not 
much bug fix activity in those areas.

Maybe http.server should be split into the "basic parts" (used by bottle.py, and other 
frameworks), and the "higher-level parts", which could then be discarded by this PEP! At 
this point, though, I'd have to agree that the whole should not be discarded. Thanks for making me 
dig deeper.

The idea has merrit. However I feel its out of scope for the PEP. The 
http.server module and socketserver module are still widely used for debug and 
toy examples.

Could you please open a bug to track your proposal? We may pursue it in a 
couple of years from now.

issue 37018

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Glenn Linderman]

Between this discussion and Steve Dower's recently referenced blog post 
at 
https://devblogs.microsoft.com/python/python-in-the-windows-10-may-2019-update/from 
which I quote below:


It’s been widely known for many years that Windows is the only 
mainstream operating system that does not include a Python interpreter 
out of the box.


and Victor's and Eric's work to make embedding Python easier, it makes 
me wonder when emacs will include a Python interpreter as an extension 
language.



On 5/22/2019 2:26 AM, Stephen J. Turnbull wrote:

Neil Schemenauer writes:

  > Here is an alternative, straw man, proposal.  Split the CPython repo
  > into two parts:
  >
  > - core Python: minimal possible stdlib
  > - everything else

I take issue with the characterization of "straw man," it's a
practical idea that turns out to be not so easy to implement.

We tried to do this with XEmacs, and found it just didn't work for us.
It requires a lot of effort to pare a distribution down to a minimal
core.  (In fact, we never got close.)  We gave up on that idea after
only a year, and this was an age and an environment that did not have
pip-like support for installation of 3rd-party packages (which you
deprecate as "not really a great solution"), and a lot of people were
still downloading at ISDN speeds or worse.  The demand for a minimal
core was correspondingly greater than today, I think.

We did bundle most applications such as MUAs and even C/C++
development support into sumo packages (one for general use, one for
packages that didn't function at all without the "MULE" multilingual
environment which was optional in XEmacs).  But we still ended up with
many packages staying in the core distribution, because other parts of
the core or the installation process used them, and as you say "If I
have a script that uses one of these modules and it gets removed, my
script breaks."  That annoyed core developers as much as anyone, and
they generally had priorities that did not emphasize minimalism (to
say the least).

  > When Python is released, provide installers for a Python that only
  > includes the "core" part and a second installer that includes
  > everything else.  I realize this is more work for the release team.
  > Hopefully with some scripting, it will not be too labour intensive.

Creating the distributions (core and sumo) was not particularly
labor-intensive for us because we mostly didn't distribute binaries.
A few moderately baroque scripts did the trick.  Automation has
advanced, but due to the amount of it it's still fragile, especially
when you're producing turn-key binary installers.

As I mentioned above, it wasn't the release process that was the
bottleneck, it was other factors that made it hard to reduce the
core.  We had an "xemacs-base" package that provided a set of "extra
tools" that some packages required, and libraries moved in and out of
it pretty much every release as some developer got tired of telling
people to install xemacs-base (back to core) or working around missing
features (move it to xemacs-base where it could be updated easily).

These are the same things that Python maintainers mention as pros and
cons of having things in stdlib or out on PyPI.  We learned there is a
sweet spot.  I know we didn't hit it most of the time, but it's very
clear that both extremes are hard, if not infeasible, to maintain.
(At the maximalist extreme, even GNU Emacs doesn't really try to
include everything written in Emacs Lisp any more.)

  > To help the people who need 100s or 1000s of extra PyPI packages,
  > we could develop a tool that creates a "sumo" Python installer,
  > grabbing packages from PyPI and building a installer package.

This could be done by any third party (it wouldn't even need to be a
public distribution, it could easily be an enterprise that open
sources an internal tool or a GSoC project).  And it could be hosted
on PyPI itself, since it's not the kind of thing that enterprises need
to install on 100,000 workstations.

  > To install that package, you would not need network access.

AIUI, it's not the network access that is usually the problem these
days, although it can be annoying if net access is flaky or if you
have to sneakernet USBs around.  Still, "sneakernet" is basically a
one-off cost.  The problem is that collections of random software from
untrusted sources are verboten in the enterprise.  (Ah, those Walnut
Creek CDs were wonderful!  I wonder what a modern AV tool would find
in them?!)  I suppose that in those situations people already have the
necessary tooling to roll the sumo or provide an intranet repository;
the problem is navigating the red tape.

  > That doesn't need to happen right away.  Also, maybe other Python
  > distributions can fill that need if core Python devs don't want to
  > build it.

I don't think this is about what core devs do or don't want to build.
I also don't feel a strong sense among the core developers that they
don't want to 

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Glenn Linderman]

On 5/21/2019 2:00 PM, Nathaniel Smith wrote:

On Tue, May 21, 2019 at 10:43 AM Glenn Linderman  wrote:

After maintaining my own version of http.server to fix or workaround some of 
its deficiencies for some years, I discovered bottle.py. It has far more 
capability, is far better documented, and is just as quick to deploy. While I 
haven't yet converted all past projects to use bottle.py, it will likely happen 
in time, unless something even simpler to use is discovered, although I can 
hardly imagine that happening.

bottle.py uses http.server for its local development mode (the one you
see in their quickstart example at the top of their README). Same with
flask, django, and probably a bunch of other frameworks. It's *very*
widely used.

-n

The source for bottle.py version 0.13-dev has an import for http.client, 
but not http.server. I hadn't tracked down every indirect dependency in 
the bottle.py source code, but it seems that if one uses the "default 
server" for bottle, that it is "wsgiref", imported from 
wsgiref.simple_server, and that in turn does import 
BaseHTTPRequestHandler and HTTPServer from http.server.


It is the higher-level code in http.server that has significant 
deficiencies that have caused me problems over the years... a 
"SimpleHTTPRequestHandler" that is so simple it doesn't do POST, PUT or 
PASTE, a "CGIHTTPRequestHandler" that only implements part of the CGI 
protocol, only CGI support in POST, no support for PUT or PASTE, and no 
support for https, and not much bug fix activity in those areas.


Maybe http.server should be split into the "basic parts" (used by 
bottle.py, and other frameworks), and the "higher-level parts", which 
could then be discarded by this PEP! At this point, though, I'd have to 
agree that the whole should not be discarded. Thanks for making me dig 
deeper.




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Glenn Linderman]

On 5/21/2019 11:15 AM, Christian Heimes wrote:

On 21/05/2019 18.29, Glenn Linderman wrote:

On 5/20/2019 2:20 PM, Christian Heimes wrote:

On 20/05/2019 23.12, Andrew Svetlov wrote:

socketserver.py is also questionable

I briefly though about the module, but didn't consider it for removal. The 
http.server, xmlrpc.server, and logging configuration server are implemented on 
top of the socketserver. I don't want to remove the socketserver module without 
a suitable replacement for http.server in the standard library.

But http.server could be on the remove list too... it gets mighty little 
support, has very little functionality, and implements a CGI interface 
(although that also has very little functionality), and you have the CGI tools 
on the remove list, rendering the CGI interface implemented by http.server less 
easily usable.

Further, it doesn't directly support https:, and browsers are removing/reducing 
support for http:.

I can't speak to xmlrpc or logging configuration.

Hi,

thanks for bringing this topic up. Initially I considered http.server, too. But 
as Guido put it, it's both used and useful for local testing and quick hacks. 
I'm already facing opposition for modules that are less controversial and 
useful than http.server, too.


Indeed. That's why they were created in the first place (http.server, 
and those other modules). But these days there are better alternatives. 
That's the point of the PEP, if I understand correctly... get rid of the 
junkware, point to the betterware, and avoid the need to learn the 
betterware later, because you wasted time and effort using the junkware 
at first, because it was "included".



I have two remarks:

1) The http.server does not act as a CGI server by default. In CGI server mode, 
it does not depend on the cgi module.


CGIHTTPRequestHandler is part of the package. While it does not depend 
on the cgi module, its existence encourages the users to search for, 
find, and use the cgi and cgitb modules when using 
CGIHTTPRequestHandler. I certainly did in past projects, prior to 
finding bottle.py.



2) The lack of HTTPS support is not a major problem for connections on localhost. There 
is a RFC draft to consider any connection to "localhost" as secure, 
https://tools.ietf.org/html/draft-west-let-localhost-be-localhost-06


Which certainly didn't exist when Chrome (noting the author's 
organization) and other browsers first started implementing feature 
restrictions for http, and while I haven't tested this since the RFC you 
mention was published, it was certainly a difficulty I faced when 
attempting to use features with security restrictions for local testing 
a couple years ago.



Christian


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Glenn Linderman]

On 5/21/2019 10:12 AM, Guido van Rossum wrote:
I still like http.server for quick temporary hacks where I want to be 
able to point a browser at some code I wrote 5 minutes ago and that I 
plan to discard in an hour. Usually it's running at localhost:8000. 
Remembering how to use Django, flask or Tornado seems overkill for 
that purpose.


After maintaining my own version of http.server to fix or workaround 
some of its deficiencies for some years, I discovered bottle.py. It has 
far more capability, is far better documented, and is just as quick to 
deploy. While I haven't yet converted all past projects to use 
bottle.py, it will likely happen in time, unless something even simpler 
to use is discovered, although I can hardly imagine that happening.


I agree that Django, flask, and Tornado have bigger learning curves, due 
to having their own philosophies on how things should be done... I 
looked at each on my way to finding bottle.py, and in each case decided 
their philosophies, while probably useful to people that like or need 
the capabilities offered, were excessive to learn if the capabilities 
were not needed.


Providing http.server as an included battery is a disservice to novice 
users, because they might try to use it for something real.


On Tue, May 21, 2019 at 9:39 AM Glenn Linderman <mailto:v%2bpyt...@g.nevcal.com>> wrote:


On 5/20/2019 2:20 PM, Christian Heimes wrote:

On 20/05/2019 23.12, Andrew Svetlov wrote:

socketserver.py is also questionable

I briefly though about the module, but didn't consider it for removal. The 
http.server, xmlrpc.server, and logging configuration server are implemented on 
top of the socketserver. I don't want to remove the socketserver module without 
a suitable replacement for http.server in the standard library.


But http.server could be on the remove list too... it gets mighty
little support, has very little functionality, and implements a
CGI interface (although that also has very little functionality),
and you have the CGI tools on the remove list, rendering the CGI
interface implemented by http.server less easily usable.

Further, it doesn't directly support https:, and browsers are
removing/reducing support for http:.

I can't speak to xmlrpc or logging configuration.




___
Python-Dev mailing list
Python-Dev@python.org <mailto:Python-Dev@python.org>
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org



--
--Guido van Rossum (python.org/~guido <http://python.org/~guido>)
/Pronouns: he/him/his //(why is my pronoun here?)/ 
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Glenn Linderman]

On 5/20/2019 2:20 PM, Christian Heimes wrote:

On 20/05/2019 23.12, Andrew Svetlov wrote:

socketserver.py is also questionable

I briefly though about the module, but didn't consider it for removal. The 
http.server, xmlrpc.server, and logging configuration server are implemented on 
top of the socketserver. I don't want to remove the socketserver module without 
a suitable replacement for http.server in the standard library.


But http.server could be on the remove list too... it gets mighty little 
support, has very little functionality, and implements a CGI interface 
(although that also has very little functionality), and you have the CGI 
tools on the remove list, rendering the CGI interface implemented by 
http.server less easily usable.


Further, it doesn't directly support https:, and browsers are 
removing/reducing support for http:.


I can't speak to xmlrpc or logging configuration.




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Easier debugging with f-strings

[Glenn Linderman]

On 5/6/2019 5:39 PM, Eric V. Smith wrote:
Last fall Larry Hastings made a suggestion for adding a way to make 
so-called "print-based debugging" easier with f-strings. Basically the 
approach is that f-strings would be able to produce the text of the 
expression and the value of that expression, without repeating the 
expression in the f-sting. No more writing f'foo={foo}, bar={bar}'. 
foo and bar should each only be in there once each!


My plan is to commit this change before 3.8b1. If anyone would like to 
discuss it at PyCon, I'll be around until about 10:30 am on Tuesday. 
I'll be in the CPython sprint room, and I'll be watching bpo, too.


Yes, I'd like this yesterday, please :)  Thanks!
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: dict.with_values(iterable)

[Glenn Linderman]

On 4/22/2019 10:59 PM, Steve Dower wrote:

On 22Apr2019 2119, Glenn Linderman wrote:
While Inada's suggested DictBuilder interface was immediately 
obvious, I don't get how either copy or update would achieve the 
goal. Perhaps you could explain? Particularly, what would be the 
trigger that would make dict() choose to create a shared key 
dictionary from the start? Unless it is known that there will be lots 
of (mostly static) dictionaries with the same set of keys at the time 
of creation of the first one, creating a shared key dictionary in 
every case would cause later inefficiencies in converting them, when 
additional items are added? (I'm assuming without knowledge that a 
single shared key dictionary is less efficient than a single regular 
dictionary.)


Passing a dictionary to the dict() constructor creates a copy of that 
dictionary (as does copy.copy()). What other trigger do you need to 
decide "it contains the same keys"? It's a copy of the original dict, 
so by definition at that point it may as well share its entire 
contents with the original.


But if the original dictionary wasn't created with shared keys... the 
copy can't share them either.  Or are you suggesting adding new code to 
create a shared key dictionary from one that isn't?


Basically this is just a partial copy-on-write, where we copy values 
eagerly - since they're almost certainly going to change - and keys 
lazily - since there are known scenarios where they are not going to 
be changed, but we'll pay the cost later if it turns out they are.


Cheers,
Steve



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: dict.with_values(iterable)

[Glenn Linderman]

On 4/22/2019 7:27 PM, Steve Dower wrote:

On 22Apr2019 1921, Steve Dower wrote:

On 22Apr2019 1822, Glenn Linderman wrote:
Inada is now proposing a way to allow the coder to suggest a group 
of dictionaries that might benefit from the same gains, by 
preclassifying non-__dict__ slot dictionaries to do similar sharing.


CSV reader is an exemplary candidate, because it creates groups of 
dicts that use the same keys. (column names). I have other code that 
does similar things, that would get similar benefits.


Seems like since it is just an interface to existing builtin code, 
that the one interface function (or dictionary factory class) could 
just as well be a builtin function, instead of requiring an import.


Sounds like a similar optimisation to sys.intern() is for strings.

I see no reason to try and avoid an import here - it's definitely a 
special-case situation - but otherwise having a function to say 
"clone and update this dict" that starts by sharing the keys in the 
same way that __dict__ does (including the transformation when 
necessary) seems like an okay addition. Maybe copy() could just be 
enabled for this?


Or possibly just "dict(existing_dict).update(new_items)".

My primary concern is still to avoid making CPython performance 
characteristics part of the Python language definition. That only 
makes it harder for alternate implementations. (Even though I was 
out-voted last time on this issue since all the publicly-known 
alternate implementations said it would be okay... I'm still going to 
put in a vote for avoiding new language semantics for the sake of a 
single runtime's performance characteristics.)


I note that dict() doesn't have a method to take two parallel iterables 
of keys/values and create a dict... if it did, that could be a trigger 
that a shared key dict might be appropriate... it seems more likely that 
data in that form is dealing with rows and columns, instead of the forms 
currently accepted by dict().


Perhaps an alternate constructor that took data in that form, AND 
defined an optional parameter to trigger a shared dict, would be a 
useful addition to the language.  Other implementations could ignore the 
optional parameter if they want, and the implementation would be a 
one-liner calling the current constructor and zip()ing the parameters.


The alternate constructor would be nice even if shared key dicts were 
not particularly needed in an application, and would provide a method of 
adding a trigger for the shared key optimization when appropriate.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: dict.with_values(iterable)

[Glenn Linderman]

On 4/22/2019 7:27 PM, Steve Dower wrote:

On 22Apr2019 1921, Steve Dower wrote:

On 22Apr2019 1822, Glenn Linderman wrote:
Inada is now proposing a way to allow the coder to suggest a group 
of dictionaries that might benefit from the same gains, by 
preclassifying non-__dict__ slot dictionaries to do similar sharing.


CSV reader is an exemplary candidate, because it creates groups of 
dicts that use the same keys. (column names). I have other code that 
does similar things, that would get similar benefits.


Seems like since it is just an interface to existing builtin code, 
that the one interface function (or dictionary factory class) could 
just as well be a builtin function, instead of requiring an import.


Sounds like a similar optimisation to sys.intern() is for strings.

I see no reason to try and avoid an import here - it's definitely a 
special-case situation - but otherwise having a function to say 
"clone and update this dict" that starts by sharing the keys in the 
same way that __dict__ does (including the transformation when 
necessary) seems like an okay addition. Maybe copy() could just be 
enabled for this?


Or possibly just "dict(existing_dict).update(new_items)".

My primary concern is still to avoid making CPython performance 
characteristics part of the Python language definition. That only 
makes it harder for alternate implementations. (Even though I was 
out-voted last time on this issue since all the publicly-known 
alternate implementations said it would be okay... I'm still going to 
put in a vote for avoiding new language semantics for the sake of a 
single runtime's performance characteristics.)


While Inada's suggested DictBuilder interface was immediately obvious, I 
don't get how either copy or update would achieve the goal. Perhaps you 
could explain? Particularly, what would be the trigger that would make 
dict() choose to create a shared key dictionary from the start? Unless 
it is known that there will be lots of (mostly static) dictionaries with 
the same set of keys at the time of creation of the first one, creating 
a shared key dictionary in every case would cause later inefficiencies 
in converting them, when additional items are added? (I'm assuming 
without knowledge that a single shared key dictionary is less efficient 
than a single regular dictionary.)


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: dict.with_values(iterable)

[Glenn Linderman]

On 4/22/2019 5:19 PM, Steven D'Aprano wrote:

Oh, you mean just like regular dicts with shared keys already do:-)

https://www.python.org/dev/peps/pep-0412/

Perhaps I've missed something in this discussion, but isn't this a
matter of just making the existing shared-keys functionality explicitly
usable rather than just purely implicit? Quoting from the PEP:

   When dictionaries are created to fill the __dict__ slot of an object,
   they are created in split form. The keys table is cached in the type,
   potentially allowing all attribute dictionaries of instances of one
   class to share keys. In the event of the keys of these dictionaries
   starting to diverge, individual dictionaries will lazily convert to
   the combined-table form.

There's no explicit interface to control this; it all happens by magic,
behind the scenes. I think the proposal here is to add some sort of
interface, possibly a new method, to explicitly use key sharing.


Thanks for the PEP reference; I'd forgotten some of the details, and 
hadn't yet gone to look them up.


Yes, it is all magic, but is only available for object __dict__ slot 
dicts. I'd forgotten that that was the "detection" mechanism.  In the 
general case, it would be too time-consuming to examine all existing 
dicts to discover some that might accidentally have the same keys, 
whereas Mark realized that objects very frequently have __dict__ slot 
dictionaries with the same keys, and were ripe for (significant memory 
and minor performance) optimization.


Inada is now proposing a way to allow the coder to suggest a group of 
dictionaries that might benefit from the same gains, by preclassifying 
non-__dict__ slot dictionaries to do similar sharing.


CSV reader is an exemplary candidate, because it creates groups of dicts 
that use the same keys. (column names). I have other code that does 
similar things, that would get similar benefits.


Seems like since it is just an interface to existing builtin code, that 
the one interface function (or dictionary factory class) could just as 
well be a builtin function, instead of requiring an import.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: dict.with_values(iterable)

[Glenn Linderman]

On 4/22/2019 4:03 PM, Inada Naoki wrote:

On Tue, Apr 23, 2019 at 2:18 AM Chris Barker via Python-Dev
 wrote:

On Fri, Apr 12, 2019 at 10:20 AM Brett Cannon  wrote:



This doesn't strike me as needing an optimization through a dedicated method.

maybe a new dict mapping type -- "shared_dict" -- it would be used in places 
like the csv reader where it makes sense, but wouldn't impact the regular dict at all.

you could get really clever an have it auto-convert to a regular dict when any 
changes were made that are incompatible with the shared keys...


My current idea is adding builder in somewhere in stdlib (maybe collections?):

   builder = DictBuilder(keys_tuple)
   value = builder(values)  # repeatedly called.

I don't want to add new mapping type because we already have shared key dict,
and changing mapping type may cause backward compatibility problem.


Regards,
As a heavy user of some self-written code that does stuff very similar 
to csv reader, and creates lots of same-key dicts, I'd be supportive of 
a performance enhancing solution here, although I haven't done a 
detailed study of where the time is currently spent.


Is the problem that the existing shared key dict isn't always detected? 
Or just that knowing in advance that it is expected to be a shared key 
dict can save the detection work?


I do know that in my code, I have a complete list of keys and values 
when I create each dict, and would be happy to tweak it to use the most 
performance technique. The above looks like a nice interface, assuming 
that values is expected to be in the same iterable order as keys_tuple 
(but is there a need for keys_tuple to be a tuple? could it be any 
iterable?).
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concurrent.futures: no type discovery for PyCharm

[Glenn Linderman]

On 4/20/2019 2:08 PM, Inada Naoki wrote:

"import typing" is slow too.

2019年4月21日(日) 1:43 Ilya Kamenshchikov >:


alright, so would an import under TYPE_CHECKING guard be an
option? like:

from typingimport TYPE_CHECKING
if TYPE_CHECKING:
 from .processimport ProcessPoolExecutor
 from .threadimport ThreadPoolExecutor

Perhaps we can have both clarity and performance.



How about:


from faketyping import TYPE_CHECKING

where faketyping.py:

TYPE_CHECKING = None


I don't know enough about how TYPE_CHECKING (or typing) is optionally 
enabled to come up with an exactly correct proposal.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (Licensing question) backport of shutil.copyfile() functionality

[Glenn Linderman]

On 3/11/2019 4:35 PM, Giampaolo Rodola' wrote:

Hello,
some time ago I contributed a couple of patches to speedup 
shutil.copy*() functions:

https://bugs.python.org/issue33671
https://bugs.python.org/issue33695
I would like to backport both functionalities so that they can be used 
on Python 2.7 and <3.8 and put it on PYPI. In order to do so I will 
basically have to copy some parts of shutil module (copytree() 
function + the unit-tests I added in BPO-33671 and a couple of other 
things). Are there constraints regarding this in terms of license? Am 
I supposed to use GPL? (I was thinking about using MIT)


Note: in this package called "zerocopy" I will probably want to expose 
other functionalities such as tee(), splice() and CopyFileEx and 
TransmitFile on Windows, so it's probably gonna be half a backport and 
half a brand new project.


Thanks.


Thanks for the contributions. I don't know about the licensing.

I wonder if you should make two packages, though... one just exactly a 
backport of the shutil speedups, and the second containing the new 
functionalities.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Summary of Python tracker Issues

[Glenn Linderman]

On 2/28/2019 2:52 PM, Terry Reedy wrote:

On 2/28/2019 5:38 PM, Glenn Linderman wrote:

On 2/28/2019 2:18 PM, Jonathan Goble wrote:
On Thu, Feb 28, 2019, 5:11 PM Terry Reedy <mailto:tjre...@udel.edu>> wrote:


    On 2/28/2019 8:07 AM, Jonathan Goble wrote:
    > On Thu, Feb 28, 2019, 8:02 AM INADA Naoki
    mailto:songofaca...@gmail.com>
    > <mailto:songofaca...@gmail.com <mailto:songofaca...@gmail.com>>>
    wrote:
    >
    >     No stats for last week?
    >
    >
    > Been missing for two weeks actually. I did not receive a 
summary on

    > either the 15th or 22nd.

    Ditto for me.  I get pydev via gmane.  Anyone missing the same 
issues

    get pydev directly by email?


I get direct emails only and stated my observation above. :-)


I confirm and concur with Jonathan's observation, by searching my 
email archive for this group list. So it is not just something about 
his particular email address... except we both do use Google Mail 
services.  I do check my Google SPAM box for the account, and it 
wasn't there either, by recollection (I don't archive the SPAM, but 
delete it ASAP).


Can someone not using a Google email address also confirm?


I effectively did when I said I access via gmane -- as a newsgroup via 
NNTP.  I am sure that the mail server sends directly to news.gmane.org 
rather than through google.


That's a whole different protocol. I don't know all the configurations 
for the server that sends the Summary messages, or how it is handled, 
but you confirm it didn't get to gnane via NNTP, and Jonathan and I 
confirm it didn't get to Google email servers, but neither one really 
confirms that it didn't get to other email servers. Google email is 
definitely different than other email servers.


There seems to be enough evidence that something went wrong somewhere, 
though, and whoever maintains that process should start investigating, 
but it would still be nice to get confirmation from a non-Google email 
recipient whether they did or did not get the Summary messages.


I wonder if there is a way to manually send them, and if the missing two 
weeks of activity can be reported... once the sending problem is 
understood and resolved.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Summary of Python tracker Issues

[Glenn Linderman]

On 2/28/2019 2:18 PM, Jonathan Goble wrote:
On Thu, Feb 28, 2019, 5:11 PM Terry Reedy > wrote:


On 2/28/2019 8:07 AM, Jonathan Goble wrote:
> On Thu, Feb 28, 2019, 8:02 AM INADA Naoki
mailto:songofaca...@gmail.com>
> >>
wrote:
>
>     No stats for last week?
>
>
> Been missing for two weeks actually. I did not receive a summary on
> either the 15th or 22nd.

Ditto for me.  I get pydev via gmane.  Anyone missing the same issues
get pydev directly by email?


I get direct emails only and stated my observation above. :-)


I confirm and concur with Jonathan's observation, by searching my email 
archive for this group list. So it is not just something about his 
particular email address... except we both do use Google Mail services.  
I do check my Google SPAM box for the account, and it wasn't there 
either, by recollection (I don't archive the SPAM, but delete it ASAP).


Can someone not using a Google email address also confirm?

Glenn
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How to update namedtuple asdict() to use dict instead of OrderedDict

[Glenn Linderman]

On 1/30/2019 8:45 PM, Raymond Hettinger wrote:

On Jan 30, 2019, at 3:41 PM, Glenn Linderman  wrote:
Would it be practical to add deprecated methods to regular dict for the OrderedDict 
reordering methods that raise with an error suggesting "To use this method, convert 
dict to OrderedDict." (or some better wording).

That's an interesting idea.  Regular dicts aren't well suited to the reordering 
operations (like lists, repeated inserts at the front of the sequence wouldn't 
be performant relative to OrderedDict which uses double-linked lists 
internally).  My instinct is to leave regular dicts alone so that they can 
focus on their primary task (being good a fast lookups).


My goal was just to give a meaningful error message if someone misses 
the implications in What's New, and has code that actually does expect 
named_tuple.as_dict to have the ordering operations.


The added, deprecated methods could be removed after a couple of versions.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How to update namedtuple asdict() to use dict instead of OrderedDict

[Glenn Linderman]

On 1/30/2019 2:32 PM, Raymond Hettinger wrote:

Now that regular dicts are ordered and compact, it makes more sense for the 
_asdict() method to create a regular dict (as it did in its early days) rather 
than an OrderedDict.

...

Option 4) Just make the change directly in 3.8,  s/OrderedDict/dict/, and be 
done will it.  This gives users the benefits right away and doesn't annoy them 
with warnings that they likely don't care about.   There is some precedent for 
this.  To make namedtuple class creation faster, the *verbose* option was 
dropped without any deprecation period.  It looks like no one missed that 
feature at all, but they did get the immediate benefit of faster import times.  
In the case of using regular dicts in named tuples, people will get immediate 
and significant space savings as well as a speed benefit.

My recommendation is Option 4 as being less disruptive and more beneficial than 
the other options.  In the unlikely event that anyone is currently depending on 
the reordering methods for the output of _asdict(), the remediation is trivially 
simple:   nt._asdict() -> OrderedDict(nt.as_dict()).

What do you all think?

Option 4 sounds good to me.

Would it be practical to add deprecated methods to regular dict for the 
OrderedDict reordering methods that raise with an error suggesting "To 
use this method, convert dict to OrderedDict." (or some better wording).
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

[Glenn Linderman]

On 11/29/2018 2:10 PM, Andrew Svetlov wrote:
Neither http.client nor http.server doesn't support compression 
(gzip/compress/deflate) at all.
I doubt if we want to add this feature: for client better to use 
requests or, well, aiohttp.
The same for servers: almost any production ready web server from PyPI 
supports compression.


What production ready web servers exist on PyPi? Are there any that 
don't bring lots of baggage, their own enhanced way of doing things? The 
nice thing about the http.server is that it does things in a 
standard-conforming way, the bad thing about it is that it doesn't 
implement all the standards, and isn't maintained very well.


From just reading PyPi, it is hard to discover whether a particular 
package is production-ready or not.


I had used CherryPy for a while, but at the time it didn't support 
Python 3, and to use the same scripts behind CherryPy or Apache CGI (my 
deployment target, because that was what web hosts provided) became 
difficult for complex scripts so I reverted to http.server with a 
few private extensions (private because no one merged the bugs I 
reported some 3 versions of Python-development-process ago; back then I 
submitted patches, but I haven't had time to keep up with the churn of 
technologies Pythondev has used since Python 3 came out, which is when I 
started using Python, and I'm sure the submitted patches have bit-rotted 
by now).


When I google "python web server" the first hit is the doc page for 
http.server, the second is a wiki page that mentions CherryPy and a 
bunch of others, but the descriptions, while terse, mostly point out 
some special capabilities of the server, making it seem like you not 
only get a web server, but a philosophy. I just want a web server. The 
last one, Waitress, is the only one that doesn't seem to have a 
philosophy in its description.


So it would be nice if http.server and http.client could get some basic 
improvements to be complete, or if the docs could point to a replacement 
that is a complete server, but without a philosophy or framework 
(bloatware) to have to learn and/or work around.


Glenn
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

[Glenn Linderman]

On 11/20/2018 10:33 PM, Nathaniel Smith wrote:

On Tue, Nov 20, 2018 at 6:05 PM Glenn Linderman  wrote:

On 11/20/2018 2:17 PM, Victor Stinner wrote:

IMHO performance and hiding implementation details are exclusive. You
should either use the C API with impl. details for best performances,
or use a "limited" C API for best compatibility.

The "limited" C API concept would seem to be quite sufficient for extensions 
that want to extend Python functionality to include new system calls, etc. (pywin32, 
pyMIDI, pySide, etc.) whereas the numpy and decimal might want best performance.

To make things more complicated: numpy and decimal are in a category
of modules where if you want them to perform well on JIT-based VMs,
then there's no possible C API that can achieve that. To get the
benefits of a JIT on code using numpy or decimal, the JIT has to be
able to see into their internals to do inlining etc., which means they
can't be written in C at all [1], at which point the C API becomes
irrelevant.

It's not clear to me how this affects any of the discussion in
CPython, since supporting JITs might not be part of the goal of a new
C API, and I'm not sure how many packages fall between the
numpy/decimal side and the pure-ffi side.

-n

[1] Well, there's also the option of teaching your Python JIT to
handle LLVM bitcode as a source language, which is the approach that
Graal is experimenting with. It seems completely wacky to me to hope
you could write a C API emulation layer like PyPy's cpyext, and
compile that + C extension code to LLVM bitcode, translate the LLVM
bitcode to JVM bytecode, inline the whole mess into your Python JIT,
and then fold everything away to produce something reasonable. But I
could be wrong, and Oracle is throwing a lot of money at Graal so I
guess we'll find out.

Interesting, thanks for the introduction to wacky. I was quite content 
with the idea that numpy, and other modules that would choose to use the 
unlimited API, would be sacrificing portability to non-CPython 
implementations... except by providing a Python equivalent (decimal, and 
some others do that, IIRC).


Regarding JIT in general, though, it would seem that "precompiled" 
extensions like numpy would not need to be re-compiled by the JIT.


But if it does, then the JIT better understand/support C syntax, but JVM 
JITs probably don't! so that leads to the scenario you describe.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

[Glenn Linderman]

On 11/20/2018 2:17 PM, Victor Stinner wrote:

Le mar. 20 nov. 2018 à 23:08, Stefan Krah  a écrit :

Intuitively, it should probably not be part of a limited API, but I never
quite understood the purpose of this API, because I regularly need any
function that I can get my hands on.
(...)
Reading typed strings directly into an array with minimal overhead.

IMHO performance and hiding implementation details are exclusive. You
should either use the C API with impl. details for best performances,
or use a "limited" C API for best compatibility.


The "limited" C API concept would seem to be quite sufficient for 
extensions that want to extend Python functionality to include new 
system calls, etc. (pywin32, pyMIDI, pySide, etc.) whereas the numpy and 
decimal might want best performance.



Since I would like to not touch the C API with impl. details, you can
imagine to have two compilation modes: one for best performances on
CPython, one for best compatibility (ex: compatible with PyPy). I'm
not sure how the "compilation mode" will be selected.


The nicest interface from a compilation point of view would be to have 
two #include files: One to import the limited API, and one to import the 
performance API. Importing both should be allowed and should work.


If you import the performance API, you have to learn more, and be more 
careful.


Of course, there might be appropriate subsets of each API, having 
multiple include files, to avoid including everything, but that is a 
refinement.




Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/v%2Bpython%40g.nevcal.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Need discussion for a PR about memory and objects

[Glenn Linderman]

On 11/19/2018 4:14 PM, Chris Barker via Python-Dev wrote:
On Mon, Nov 19, 2018 at 1:41 AM Antoine Pitrou > wrote:


I'd rather keep the reference to memory addressing than start
doing car
analogies in the reference documentation.


I agree -- and any of the car analogies will probably be only valid in 
some jurisdictions, anyway.


I think being a bit more explicit about what properties an ID has, and 
how the id() function works, and we may not need an anlogy at all, 
it's not that difficult a concept. And methions that in c_python the 
id is (currently) the memory address is a good idea for those that 
will wonder about it, and if there is enough explanation, folks that 
don't know about memory addresses will not get confused.


This is what's in the docs now (3.8.0a0):

"""
Every object has an identity, a type and a value. An object’s identity 
never changes once it has been created; you may think of it as the 
object’s address in memory. The ‘is’ operator compares the identity of 
two objects; the id() function returns an integer representing its 
identity.


**CPython implementation detail:** For CPython, id(x) is the memory 
address where x is stored.

"""

I suggest something like the following:

"""
Every object has an identity, a type and a value. An object’s identity 
uniquely identifies the object. It will remain the same as long as 
that object exists. No two different objects will have the same id at 
the same time, but the same id may be re-used for future objects once 
one has been deleted. The ‘is’ operator compares the identity of two 
objects; the id() function returns an integer representing its 
identity. ``id(object_a) == id(object_b)`` if and only if they are the 
same object.


**CPython implementation detail:** For CPython, id(x) is the memory 
address where x is stored.

"""



Well re-worded in my opinion.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Standardize error message for non-pickleable types

[Glenn Linderman]

On 10/30/2018 1:12 AM, Serhiy Storchaka wrote:

29.10.18 23:17, MRAB пише:

1. If you're pickling, then saying "pickle" is more helpful.

2. In English the usual long form is "cannot". Error messages tend to 
avoid abbreviations, and also tend to have lowercase after the colon, 
e.g.:


 "ZeroDivisionError: division by zero"

 "ValueError: invalid literal for int() with base 10: 'foo'"

3. If it's failing on an object (singular), then it's clearer to say 
"object".


4. Articles tend to be omitted.

5. Error messages tend to have quotes around the type name.

Therefore, my preference is for:

 "cannot pickle 'XXX' object"


Thank you Matthew, I'll use your variant.

Will something change the fact that in all these cases the pickling 
will be failed not just for specific object, but for all instances of 
the specified type?
That's why I suggested "object of type 'XXX'", to leave the type in a 
more prominent position, as it is generally more important to the issue 
than the object.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] short-circuiting runtime errors/exceptions in python debugger.

[Glenn Linderman]

On 10/29/2018 3:07 PM, Greg Ewing wrote:

When I have a bug that only happens after hours of run
time, I try to find a much shorter test case that reproduces
it.


Mmm. Yeah.  But that's often a guessing game, with a low chance of success.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Standardize error message for non-pickleable types

[Glenn Linderman]

On 10/29/2018 12:51 PM, Victor Stinner wrote:

Le lun. 29 oct. 2018 à 20:42, Serhiy Storchaka  a écrit :

1. "pickle" or "serialize"?

serialize


2. "can't", "Cannot", "can not" or "cannot"?

cannot


3. "object" or "objects"?

object


4. Use the "a" article or not?

no: "cannot serialize xxx object" (but i'm not a native english
speaker, so don't trust me :-))


Cannot serialize an object of type 'XXX'




5. Use quotes around type name or not?

Use repr() in Python, but use '%s' is C since it would be too complex
to write code to properly implement repr() (decode tp_name from UTF-8,
handle error, call repr, handle error, etc.).

To use repr() on tp_name, I would prefer to have a new formatter, see
the thread of last month.
https://mail.python.org/pipermail/python-dev/2018-September/155150.html

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/v%2Bpython%40g.nevcal.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Arbitrary non-identifier string keys when using **kwargs

[Glenn Linderman]

On 10/9/2018 7:46 PM, Chris Jerdonek wrote:

On Tue, Oct 9, 2018 at 7:13 PM Benjamin Peterson  wrote:

On Tue, Oct 9, 2018, at 17:14, Barry Warsaw wrote:

On Oct 9, 2018, at 16:21, Steven D'Aprano  wrote:

On Tue, Oct 09, 2018 at 10:26:50AM -0700, Guido van Rossum wrote:

My feeling is that limiting it to strings is fine, but checking those
strings for resembling identifiers is pointless and wasteful.

Sure. The question is, do we have to support uses where people
intentionally smuggle non-identifier strings as keys via **kwargs?

I would not be in favor of that.  I think it doesn’t make sense to be
able to smuggle those in via **kwargs when it’s not supported by
Python’s grammar/syntax.

Can anyone think of a situation where it would be advantageous for an 
implementation to reject non-identifier string kwargs? I can't.

One possibility is that it could foreclose certain security bugs from
happening. For example, if someone has an API that accepts **kwargs,
they might have the mistaken assumption that the keys are identifiers
without special characters like ";" etc, and so they could make the
mistake of thinking they don't need to escape / sanitize them.

--Chris
With that line of reasoning, one should make sure the data are 
identifiers too :)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] why is not 64-bit installer the default download link for Windows?

[Glenn Linderman]

On 7/10/2018 11:14 PM, Stephen J. Turnbull wrote:

Ivan Pozdeev via Python-Dev writes:

  > "One or more issues caused the setup to fail. Please fix the issues and
  > the retry setup. For more information see the log file .
  >
  > 0x80070661 - This installation package is not supported by this
  > processor type. Contact your product vendor."
  >
  > Pretty descriptive in my book.

Experience shows that's definitely not descriptive enough for my
university's students (and starting from AY 2021 we're moving to
Python 3 as the university-wide programming course language, yay!)
They have no idea that "processor type" means "word size", or what
alternative package to look for.  Sometimes they take the "contact
vendor" wording to mean "package is broken".  I don't think the
Japanese or Chinese students will have 32-bit machines (haven't seen
one among my advisees since March 2016), but we do get some students
from less wealthy countries who may be using older machines.

So I think it would be really nice if the installer detects the
wordsize mismatch, and issues an explicit message like

 This package is intended for a 64-it machine, but yours is a 32-bit
 machine.

 Please download and install the package specifically for 32-bit
 machines instead.
Which would be far, far better, regardless of which bitness(es) of 
installer is(are) displayed (prominently) on the web site.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 572: Assignment Expressions -- intention to accept, near-final draft

[Glenn Linderman]

On 7/9/2018 8:43 PM, Guido van Rossum wrote:



An exception to this special case applies when the target name is the
same as a loop control variable for a comprehension containing it.
This is invalid.  This exception exists to rule out edge cases of the
above scope rules as illustrated by ``[i := i+1 for i in range(5)]``
or ``[[(j := j) for i in range(5)] for j in range(5)]``. Note that
this exception also applies to ``[i := 0 for i, j in stuff]``, as
well
as to cases like ``[i+1 for i in i := stuff]``.



It is unclear whether exactly what is invalid. Is the use of the
target name that is the same as (any of the nested) loop control
variable invalid? I think, from discussions, that that is what is
meant. But this paragraph could be interpreted as meaning the
special case doesn't apply, meaning that the target name would be
in a "sublocal" scope.


Really? If it didn't say "this is invalid" I could see that "exception 
to the special case" might be interpreted as "the special case doesn't 
apply". But with "This is invalid" explicitly added I don't see how 
that interpretation could be valid. Is it clearer if I changed that to 
"Such code is invalid"? Or perhaps I should move "This is invalid" to 
the end of the paragraph?


That's better; even better might be to say what is invalid... instead of 
used pronoun.


"Use of any of the comprension loop control variables as a target name 
in an assignment expression is invalid."


It isn't really an exception to the special case, it is an exception to 
the general rule than any old name can be used as an assignment 
expression target, IIUC.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 572: Assignment Expressions -- intention to accept, near-final draft

[Glenn Linderman]

On 7/9/2018 6:00 PM, Guido van Rossum wrote:

This rule is included to simplify the choice for the user between an
  assignment statements and an assignment expression -- there is no


"statements" should not be plural in the above line.


syntactic position where both are valid.




An exception to this special case applies when the target name is the
same as a loop control variable for a comprehension containing it.
This is invalid.  This exception exists to rule out edge cases of the
above scope rules as illustrated by ``[i := i+1 for i in range(5)]``
or ``[[(j := j) for i in range(5)] for j in range(5)]``.  Note that
this exception also applies to ``[i := 0 for i, j in stuff]``, as well
as to cases like ``[i+1 for i in i := stuff]``.



It is unclear whether exactly what is invalid. Is the use of the target 
name that is the same as (any of the nested) loop control variable 
invalid? I think, from discussions, that that is what is meant. But this 
paragraph could be interpreted as meaning the special case doesn't 
apply, meaning that the target name would be in a "sublocal" scope.




The ``:=`` operator groups more tightly than a comma in all syntactic
positions where it is legal, but less tightly than all operators,


If comma is considered an operator, this sentence is inconsistent, would 
need to be "all other operators".  Even if comma is not considered an 
operator, the sentence would be more clear with "other" added, since 
":=" is an operator.



Sorry, I haven't learned pull requests.
Glenn

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Naming comprehension syntax [was Re: Informal educator feedback on PEP 572 ...]

[Glenn Linderman]

On 7/6/2018 9:01 PM, Terry Reedy wrote:
In any case, Python's comprehensions use an English-syntax version of 
extended set builder notation.  "In Python, the set-builder's braces 
are replaced with square brackets, parentheses, or curly braces, 
giving list, generator, and set objects, respectively. Python uses an 
English-based syntax."


Also, "generator builder" is not much more expressive than 
"generator expression",


I looked for an alternative 'x' to 'comprehension' such that 
'generator|list|set|dict x' works and is specific to the notation. 
'Builder' is a reasonable choice. 


I'm not sure if your quote above was quoting documentation, or was a 
suggested quote to add to the documentation, I think the latter, as 
Google didn't find it.


The conflict between the "Builder pattern" and "set-builder notation" 
can be disambiguated by consistently using the hyphenated "set-builder" 
(as wikipedia does). And happily, by using wikipedia terms, they would 
be easily found with explanations outside of python docs as well as (if 
this is done) inside.  We do not need


[ typ + ' builder' for typ in ('set', 'list', 'dict', 'generator')]

only set-builder.  The fencing and : determine the type of the result.  
We could use


[ typ + ' form of set-builder'  for typ in ('set', 'list', 'dict', 
'generator')]


in the few places where the type of the set-builder must be 
disambiguated, avoiding the need for the compound terms.


The result of  ( set-builder ) is a generator. We do not need the term 
"generator expression" or "generator comprehension".  Use "generator 
form of set-builder"... yes, it is one or two syllables longer, but is 
clearer.


A generator can be produced in one of two ways: either a function 
containing a yield, or a set-builder delimited by parentheses or used as 
an actual parameter to a function, both of which can be referred to as 
the "generator form of set-builder".


Glenn
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Naming comprehension syntax [was Re: Informal educator feedback on PEP 572 ...]

[Glenn Linderman]

On 7/6/2018 11:20 AM, Brett Cannon wrote:



On Fri, 6 Jul 2018 at 08:52 Chris Barker - NOAA Federal via Python-Dev 
mailto:python-dev@python.org>> wrote:


Are we just having fun here?

Or might we actually start using a new naming convention for
the-syntax-formerly-known-as-generator-expressions?


If you can create a groundswell of support then maybe.


"Comprehension" was an incomprehensible term to me, when I first heard 
it. After reading the documentation, I comprehended it, but the term is 
meaningless.


"generator expression" is actually a little more comprehensible, by 
analogy to a power (electricity) generator... values are generated. But 
comprehensions would have been more intuitively understood by me, had 
they been called "list generators" or "dict generators" or "set generators".


The difference between comprehensions and generators seems to be one of 
timing and sequencing: the comprehensions are comprehended as fast as 
possible, while generators are lazy (unlike those electricity 
generators, unless you turn them on and off repeatedly). So neither term 
is very intuitive to "plain old programmers" unless perhaps they happen 
to know some of the advanced math concepts where the term 
"comprehension" is claimed to come from (obviously, in spite of a math 
degree from some years back, I never encountered that term).


Focusing just on lists, at this time, "list builder" would be better 
than "list generator" because using "generator" to replace 
"comprehension" would be confusing because of the history of the 
terminology as used/documented currently.


A generator seems to be a "lazy list builder".

If the names are changed, for a time both terminologies would have to 
coexist, so it would be critical to use terms not already in use in 
Python terminology. I would find it an improvement to use terms like 
"list builder" rather than "comprehension".




-Brett


-CHB

Sent from my iPhone

> On Jul 3, 2018, at 11:54 PM, Greg Ewing
mailto:greg.ew...@canterbury.ac.nz>>
wrote:
>
> Steven D'Aprano wrote:
>> - list builder syntax is syntax which returns a list;
>> - dict builder syntax is syntax which returns a dict;
>> - set builder syntax is syntax which returns a set;
>> - generator builder syntax is syntax which returns a generator.
>
> You only get a list/dict/set from the first three after
> you've run the iterators within it, but with a generator
> expression, you already have a generator before you've
> run it. That makes it feel different to me.
>
> --
> Greg



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Slow down...

[Glenn Linderman]

On 5/7/2018 7:59 AM, Eric Snow wrote:

On Sun, May 6, 2018 at 8:25 PM, Nick Coghlan  wrote:

I'm inclined to agree that a Python 3.8 PEP in the spirit of the PEP 3003
language moratorium could be a very good idea.

Note that the PEP specifically applies to "syntax, semantics, and
built-ins".  Here's the full abstract [1]:

 This PEP proposes a temporary moratorium (suspension) of all changes to the
 Python language syntax, semantics, and built-ins for a period of
at least two
 years from the release of Python 3.1. In particular, the moratorium would
 include Python 3.2 (to be released 18-24 months after 3.1) but allow Python
 3.3 (assuming it is not released prematurely) to once again include 
language
 changes.

 This suspension of features is designed to allow non-CPython 
implementations
 to "catch up" to the core implementation of the language, help ease 
adoption
 of Python 3.x, and provide a more stable base for the community.

-eric

Here's my "lightning" response to a "lightning talk" about a moratorium:

So if other implementations didn't catch up during the last moratorium, 
either the moratorium then was lifted too soon, or the other 
implementations don't really want to catch up, or the thought that they 
should catch up was deemed less important than making forward progress 
with the language. Have any of those opinions changed?


While async is a big change that I personally haven't grasped, but which 
has little impact (a couple keywords) on code that doesn't use it, a new 
moratorium wouldn't impact it, and nothing else that is happening seems 
too much or too fast from my perspective.


Dan's original comment about language versus library is interesting, 
though. It is probably true that one should resist adding language where 
library suffices, but sometimes a lack of certain expressiveness in the 
language causes cumbersome implementations of library routines to 
achieve the goal.


f-strings and binding expressions are cases where (1) the functionality 
is certainly possible via library (2) there is a large amount of code 
that uses the functionality, and (3) said code is more cumbersome 
without the expressiveness of the newer syntax.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A fast startup patch (was: Python startup time)

[Glenn Linderman]

On 5/5/2018 10:30 AM, Toshio Kuratomi wrote:
On Fri, May 4, 2018, 7:00 PM Nathaniel Smith > wrote:


What are the obstacles to including "preloaded" objects in regular
.pyc files, so that everyone can take advantage of this without
rebuilding the interpreter?


Would this make .pyc files arch specific?


Lots of room in the __pycache__ folder.

As compilation of the .py module proceeds, could it be determined if 
there is anything that needs to be architecture specific, and emit an 
architecture-specific one or an architecture-independent one as 
appropriate?  Data structures are mostly bitness-dependent, no?


But if an architecture-specific .pyc is required, could/should it be 
structured and named according to the OS conventions also:  .dll .so  .etc ?


Even if it doesn't contain executable code, the bytecode could be 
contained in appropriate data sections, and there has been talk about 
doing relocation of pointer in such pre-compiled data structures, and 
the linker _already_ can do that sort of thing...
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python startup time

[Glenn Linderman]

On 5/2/2018 8:56 PM, Gregory Szorc wrote:
Nobody in the project is seriously talking about a complete rewrite in 
Rust. Contributors to the project have varying opinions on how 
aggressively Rust should be utilized. People who contribute to the C 
code, low-level primitives (like storage, deltas, etc), and those who 
care about performance tend to want more Rust. One thing we almost 
universally agree on is that we want to rewrite all of Mercurial's C 
code in Rust. I anticipate that figuring out the balance between Rust 
and Python in Mercurial will be an ongoing conversation/process for 
the next few years.

Have you considered simply rewriting CPython in Rust?

And yes, the 4th word in that question was intended to produce peals of 
shocked laughter. But why Rust? Why not Go? http://esr.ibiblio.org/?p=7724
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20

[Glenn Linderman]

On 4/27/2018 2:11 PM, Tim Peters wrote:

That's easy:  any time there's a long thread to which Guido has
contributed at least twice, it will be seen as a Golden Opportunity to
re-litigate every decision that's ever been made;-)

You're getting pretty good at that QOTD thing, Tim :)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com