[Python-Dev] Re: Possible bug in `re` module in Python 3.11?

[Serhiy Storchaka]

26.10.22 11:17, Piotr Waszkiewicz пише:

Hi,
I would like to ask your guidance as I'm entirely sure whether the problem I'm 
experiencing should be posted in CPython's repo as a bug issue.
I've tried using newly released Python 3.11 interpreter in some of my projects and one of 
them failed to start with "RuntimeError: invalid SRE code" error.

Looking into implementation of one of the dependencies I've found out that the 
issue has been found out by a maintainer and fixed 
(https://github.com/pydicom/pydicom/issues/1658).
It looks like this particular regexp caused the `re.compile()` method to raise:

```
re.compile(
 r"(?P^([01][0-9]|2[0-3]))"
 r"((?P([0-5][0-9]))?"
 r"(?(5)(?P([0-5][0-9]|60))?)"
 r"(?(7)(\.(?P([0-9]{1,6})?))?))$"
 )
```

I've checked and this hasn't been an issue in all previous Python interpreter 
versions, starting from 3.6 (the oldest I've checked).
What's more the regex i correctly recognized and does not cause any issues in 
other regexp implementations, e.g. the online tool https://regex101.com/

Could somebody help me decide whether this is indeed a bug?


Yes, it is a bug, and I have found its cause. Please open an issue on 
the CPython bug tracker.




___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GQP6P2CLVX4EOND73JYOXWNCIA3HIERW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Adding new escapes to regex module

[Serhiy Storchaka]

17.08.22 19:34, MRAB пише:
It's not just Java. Perl supports all 4 of \h, \H, \v and \V. That might 
be why Java 8 changed.


But Perl does not have conflict between strings and regular expressions, 
because regular expression is a separate syntax construct.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/V47KWBKWCTBBXM637VPPJTR3QD4C5S23/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Adding new escapes to regex module

[Serhiy Storchaka]

16.08.22 23:24, MRAB пише:
Other regex implementations have escape sequences for horizontal 
whitespace (`\h` and `\H`) and vertical whitespace (`\v` and `\V`).


The regex module already supports `\h`, but I can't use `\v` because it 
represents `\0x0b', as it does in the re module.


Now that someone has asked for it, I'm trying to find a nice way of 
adding it, and I'm currently thinking that maybe I could use `\y` and 
`\Y` instead as they look a little like `\v` and `\V`, and, also, 
vertical whitespace is sort-of in the y-direction.


As far as I can tell, only ProgressSQL uses them, and, even then, it's 
for what everyone else writes as `\b` and `\B`.


I want the regex module to remain compatible with the re module, in case 
they get added there sometime in the future.


Opinions?


I do not like introducing escapes which are not supported in other RE 
implementations. There is a chance of future conflicts.


Java broke compatibility in Java 8 by redefining \v from a single 
vertical tab character to the vertical whitespace class. I am not sure 
that it is a good example that we should follow, because different 
semantic of \v in raw and non-raw strings is a potential source of bugs. 
But with special flag which controls the meaning of \v it may be more safe.


Horizontal whitespace can be matched by [ 
\t\xA0\u1680\u180e\u2000-\u200a\u202f\u205f\u3000] in re or [\t\p{Zs}] 
in regex. Vertical whitespace can be matched by 
[\n\x0b\f\r\x85\u2028\u2029]. Note that there is a dedicated Unicode 
category for horizontal whitespaces (excluding the tab itself), but not 
for vertical whitespaces, it means that vertical whitespaces are less 
important.


In any case it is simple to introduce special Unicode categories and use 
\p{ht} and \p{vt} for horizontal and vertical whitespaces.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CXK6TDJDGYM2BTPVNIDQIWMQN76K65KN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Raw strings ending with a backslash

[Serhiy Storchaka]

28.05.22 18:03, Damian Shaw пише:
My understanding was that was part of the question being asked, is it 
possible to know what with the new PEG parser?


You first need to define what is the end of a string. And I think it is 
not relevant to the grammar parser.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I5FLGQ3KVDXVITFTW6LO2KADODKTGZ5O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Raw strings ending with a backslash

[Serhiy Storchaka]

28.05.22 14:57, Damian Shaw пише:

That PR seems to make \' and \" not special in general right?

I think this is a more limited proposal, to only change the behavior 
when \ is at the end of a string, so the only behavior difference would 
never receiving the error "SyntaxError: EOL while scanning string literal"


In which case there should be no backwards compatibility issue.


How do you know that it is at the end of a string?

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BFGT3H57CFTEOKWA3NXBSPUDE4JF4C2H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Raw strings ending with a backslash

[Serhiy Storchaka]

28.05.22 12:22, Steven D'Aprano пише:

Now that we have a new parser for CPython, can we fix the old gotcha
that raw strings cannot end in a backslash?

Its an FAQ and has come up again on the bug tracker.

https://docs.python.org/3/faq/design.html#id26

https://github.com/python/cpython/issues/93314


I do not think that we can allow this, and it is not related to parser.

Few years ago I experimented with such change:
https://github.com/python/cpython/pull/15217

You can see that it breaks even some stdlib code, and it will definitely 
break many third-party packages and examples. Technically we can do 
this, but the benefit is small in comparison with the cost.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WWWRFQK4AG52GD3L6WT6QLRGTY2VQRQ2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Starting a new thread

[Serhiy Storchaka]

10.05.22 18:12, Barney Stratford пише:

This has a couple of advantages. I don’t have to import the threading module 
all over my code. I can use the neater syntax of function calls. The function’s 
definition makes it clear it’s returning a new thread since it’s decorated. It 
gets the plumbing out of the way so I an concentrate more on what my code does 
and less in how it does it.


I do not see advantages. You definitely need to import the threading 
module or other module depending on the threading module in which you 
define the decorator. And you need to decorate the function. It will not 
save you a line of code.


If you need to run a lot of functions in threads, note that a thread has 
some starting cost. Consider using concurrent.futures.ThreadPoolExecutor.


If you only run few long living threads, the syntax of starting them 
does not matter, in comparison with the rest of your code.


Also, it looks wrong to me to mix two different things: what code to 
execute and how to run it. If we need a neater syntax, I would propose:


t = threading.start_thread(func, *args, **kwargs)

But I am not sure that it is worth to add such three-line function in 
the stdlib.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RBIAWP2J3NTYMLIS3W6CKX44G5QIBXAU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python grammar test cases

[Serhiy Storchaka]

10.05.22 08:10, Venkat Ramakrishnan пише:

I'm wondering if there's a repository of test cases that
test the Python grammar. Any help would be appreciated.


See test_grammar and test_syntax. And there are some test files for 
specific grammar features, like test_string_literals, test_unpack_ex, 
test_genexps, test_fstring, etc.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NW2D3WMQOOFVF5WHJDEG5B4TCIY7UR5U/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: __dunder__ = None

[Serhiy Storchaka]

01.05.22 19:32, Guido van Rossum пише:
Non should behave as closely as possible to it not being defined at all. 
So return NotImplemented.


But, as was noted by Spencer Brown, it is not how it works for 
__iter__(), __reversed__() and __contains__() (which have a special path 
for producing explicit error), and not how it currently works for 
__add__() and __eq__() (although they raise TypeError implicitly). It is 
all documented inhttps://docs.python.org/3.10/reference/datamodel.html#id2


If we are going to reconsider this, it is a large breaking change.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/V5XNDNA3I3C5VAARBDSSF72ILYWKVI2M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: __dunder__ = None

[Serhiy Storchaka]

01.05.22 13:02, Spencer Brown пише:
It actually is documented as being supported here: 
https://docs.python.org/3.10/reference/datamodel.html#id2 
, and as 
mentioned there __iter__(), __reversed__() and __contains__() also have 
special support so they avoid trying to fallback to __getitem__ etc. 
Also some discussion at https://github.com/python/cpython/issues/70146 
. For most methods the 
conclusion back then makes sense, a specific exception message being 
raised would just potentially slow the interpreter.


Thank you Spencer. It answers my questions.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LNE2COY3OAV7EJOMPBFBHSVPBTYM4KUQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] __dunder__ = None

[Serhiy Storchaka]

There is a special handling of `__hash__` set to None in the interpreter 
core. This is because every class inherited the `__hash__` attribute 
from "object", and setting `__hash__ = None` is a simple way to make it 
unhashable. It makes hash() raising the correct type of exception 
(TypeError), but with unhelpful error message "'NoneType' object is not 
callable". The special case was added to make the error message more 
relevant: "unhashable type: '{typename}'".


There is similar situation with other special methods defined in 
"object" or other common classes. Sometimes we want to cancel the 
default inherited behavior.


   >>> dir(object)
   ['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', 
'__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', 
'__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', 
'__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', 
'__setattr__', '__sizeof__', '__str__', '__subclasshook__']


I propose to support officially the idiom "__dunder__ = None" and add 
special cases to raise more specialized exception instead of "TypeError: 
'NoneType' object is not callable" for most of special method where 
cancelling the default behavior makes sense (for example I do not think 
that we need better error message for `__repr__ = None`).


The question is how to interpret value None:

* Always raise TypeError (with changed message)? This is what happen 
currently when you set the method to None, this is the most compatible 
option.
* Always raise an error, but allow to change it to more appropriate type 
(for example AttributeError for __setattr__)?

* Interpret value None the same way as an absent attribute?

For `__hash__` or `__class_getitem__` all three options mean the same. 
But absent `__mro_entries__` and `__mro_entries__ = None` currently give 
different results. It is even more complicated for pickling: absent 
`__reduce_ex__` and `__reduce_ex__ = None` mean the same in the Python 
implementation, but give different results in the C implementation.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YGAK34DRWJFSIV2VZ4NC2J24XO37GCMM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Restrict the type of __slots__

[Serhiy Storchaka]

18.03.22 11:29, Serhiy Storchaka пише:
It will break some code (there are 2 occurrences in the stdlib an 1 in 
scripts), but that code can be easily fixed.


Actually it will break more code, because initializing __slots__ with a 
list is pretty common. Maybe restrict it to tuple or list? But having an 
immutable __slots__ may be more reliable.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HCZHJN2LSD2NXFLSFAO7VVOMEZRTLDBQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Restrict the type of __slots__

[Serhiy Storchaka]

Currently __slots__ can be either string or an iterable of strings.

1. If it is a string, it is a name of a single slot. Third-party code 
which iterates __slots__ will be confused.


2. If it is an iterable, it should emit names of slots. Note that 
non-reiterable iterators are accepted too, but it causes weird bugs if 
__slots__ is iterated more than once. For example it breaks default 
pickling and copying.


I propose to restrict the type of __slots__. Require it always been a 
tuple of strings. Most __slots__ in real code are tuples. It is rarely 
we need only single slot and set __slots__ as a string.


It will break some code (there are 2 occurrences in the stdlib an 1 in 
scripts), but that code can be easily fixed.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/E32BRLAWOU5GESMZ5MLAOIYPXSL37HOI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Make __mro_entries__ mandatory for non-types

[Serhiy Storchaka]

Currently the class can inherit from arbitrary objects, not only types. 
If the object was not designed for this you can get a mysterious 
mistake, e g.:


>>> class A(1): ...
...
Traceback (most recent call last):
  File "", line 1, in 
TypeError: int() takes at most 2 arguments (3 given)

If the object's constructor is compatible with type constructor by 
accident you can an unexpected result. To prevent this we usually add an 
explicit __mro_entries__() method which raises an exception.


I propose to make __mro_entries__() mandatory if a base is not a type 
instance (with a deprecation period).


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QDFG65U5ILB7LISC7UL2WFGQIAL2DMUV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Require a C compiler supporting C99 to build Python 3.11

[Serhiy Storchaka]

07.02.22 19:09, Victor Stinner пише:
> After my NAN change (bpo-46640), Petr Viktorin asked me to update the
> PEP 7. I proposed a change to simply say that "Python 3.11 and newer
> versions use C99":

But public headers should be compatible with C++, and not all C99
features are compatible with C++. It is not a matter of style, it is a
matter of compatibility.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZDGZN46UQAPJY5EF5A3GYA5DT4XX6AQO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Replace debug runtime checks in release mode with assertions in debug mode

[Serhiy Storchaka]

07.02.22 17:55, Victor Stinner пише:
> I'm asking to replace runtime checks with assertions when the C API is
> "obviously" misused: replace PyErr_BadInternalCall(),
> _Py_CheckFunctionResult() and _Py_CheckSlotResult() with assertions.
> The exact scope should be defined.

Would not be better to call Py_FatalError() in PyErr_BadInternalCall()
etc? It can even be controlled by some option or envvar, even in release
mode (just with different defaults in debug and release modes).

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Q6JOWJND63URG3Y2WAAM7QJCCKNFUQAE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Suggestion: a little language for type definitions

[Serhiy Storchaka]

08.01.22 01:59, jack.jan...@cwi.nl пише:
>> If I can make a wild suggestion: why not create a little language for
>> type specifications?

We need a way to define aliases. For example, write:

Data = Mapping[str, Sequence[Tuple[int, T]]]
Factory = Callable[[int, Iterable[str]], Optional[list[Data[T

def get_foo_factory(type: str, id: int) -> Factory[Foo]: ...

instead of

def get_foo_factory(type: str, id: int) -> Callable[[int,
Iterable[str]], Optional[list[Mapping[str, Sequence[Tuple[int,
Foo]]: ...

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/42A5UC246FL2C57WSXP5Y2DANFGTG4PU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Serhiy Storchaka]

24.12.21 00:09, Guido van Rossum пише:
> Without decorator too (that was Lukasz’ idea). Why bother with the
> decorator (*if* we were to go there)?

It is errorprone.

Some library provide function foo() which returns an instance of private
type _Foo and people start using it as a type hint. A new version
converts foo() into a class. It is usually a safe backward compatible
change, except that now all type hints became wrong. "a: foo" now means
an instance of foo, not a callable which returns an instance of _Foo.

There are also issues with subclassing.

>>> foo = Callable[[int], str]
>>> class A(foo): ...
...
>>> def bar(x: int, /) -> str: pass
...
>>> class B(bar): ...
...
Traceback (most recent call last):
  File "", line 1, in 
TypeError: function() argument 'code' must be code, not str

There are also issues with subscripting:

>>> T = TypeVar('T')
>>> foo = Callable[[T], list[T]]
>>> list[foo][int]
list[typing.Callable[[int], list[int]]]
>>> def bar(x: T, /) -> list[T]: pass
...
>>> list[bar][int]
Traceback (most recent call last):
  File "", line 1, in 
TypeError: There are no type variables left in list[__main__.bar]

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ALW5PMZ3MRIG3BGTX5DVIKFO4JS45MBK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The python command help is too long

[Serhiy Storchaka]

20.12.21 21:34, Guido van Rossum пише:
> Fair enough. Quick design proposal:
> 
>   -h and --help print info about flags (existing flags)
>   --help-env (or --env-help?) prints info about env vars (new flag)
>   -X help prints info about -X options (new -X option)
> 
> Two lines printed at the end of the -h/--help output would refer users
> to --help-env and -Xhelp.

Thank you Guido and Barry for your suggestions. It looks great!

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7PL3GAK26HXX3PLFU6MSHNUHKOYE3D7G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Serhiy Storchaka]

20.12.21 21:28, Brett Cannon пише:
> As someone with use of this, would you find this useful (i.e. +1, +0)?
> Serhiy already said "no" in another thread.

In every file we import 5-10 or more names from the typing module. We
still does not use PEP 585 and PEP 604 syntax, but are going to do this
in near future. It could save as from importing List, Tuple, Dict, Set,
Union, Optional, but we still need to import Any, Sequence, Iterable,
Iterator, AsyncIterator, Awaitable, TypeVar, and sometimes
AsyncContextManager, NewType, cast, overload. Special syntax for
callable type hints will not save game.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NLLN4IGOKCQZP7LYM7VBQSGGGWPW3WKZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Serhiy Storchaka]

20.12.21 13:42, Mark Shannon пише:
> OOI, of those 1577 Callable type hints, how many distinct Callable types?

Around 15-20%. Most of them are in tests which widely use pytest
fixtures, so functions taking and returning callables are common. There
are around 200 occurrences in non-test code, half of them are distinct
types.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7YXSPACKOU7AGOHEZX2VAMXEELA3OZGA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Serhiy Storchaka]

18.12.21 23:07, Terry Reedy пише:
> Batuhan expresses my concerns better than I could, so I just add my
> agreement.
> 
> On 12/18/2021 3:13 PM, Batuhan Taskaya wrote:
> 
>> tl;dr: I find it very troubling that we are going on a path where need
>> to increase the language complexity (syntax) only in the cause
>> 'easier' typing. So I am opposed to this change.

I concur.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DVREYSY6BIPCESCOT4QP2SV5CPG53LXB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] The python command help is too long

[Serhiy Storchaka]

The output of "python -h" is 104 lines long now. It was only 51 lines in
3.6. 35% of it is about the -X option, and 30% about environment
variables. Also some lines in the -X option description are too long
(102 columns). Both topics are "advanced" and mostly interested for
debugging. I suggest to move them out of the main help output.

The are two options:

1. Move it to pydoc topics. The advantage is that it is a standard way,
there are already 88 topics. The disadvantage is that this information
will be not available in "minimal" installations of Python which do not
include docs.

2. Add command-line options like -hX and -henv. The information will
always be available with the interpreter, but the interface is special.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QUUBM7DGSXYWBOLZNWOSCQUDALWJIYZF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: multiple inheritance and `__str__`

[Serhiy Storchaka]

01.12.21 09:56, Ethan Furman пише:
> Some searching turned up issue 36793: "Do not define unneeded __str__
> equal to __repr__" .
> 
> As can be seen, `__str__` is needed for inheritance to work properly. 
> Thoughts on reverting that patch?

As the author of issue36793 I do not think so. If you replace int with
list you will get the same result. If you make Blah an int subclass you
will get the same result. If you want to get different results set
Huh.__str__ explicitly to object.__str__ or int.__repr__.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/U3VVGYXXQ4RED64SLCX63BJITCCVLJFL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Optimizing literal comparisons and contains

[Serhiy Storchaka]

29.11.21 18:36, Victor Stinner пише:
> You should consider "no longer have to justify why it's not optimized"
> as a clear benefit of making this change :-) This optimization is
> proposed once a year for many years...
> 
> For me, any possible compilation-ahead optimization (which doesn't
> break the Python semantics) is worth it ;-) It's done once, possible
> even before the program is run for the first time, and then makes the
> code faster.

I consider the cost of rejecting such idea (this is actually the first
time it was pushed so persistent) less than the cost of implementing and
maintaining it. The proposed PR adds around 200 lines of code. It is
almost 20% of the current size of ast_opt.c, it is not small. And just
at first look I see a flaw in it -- it will emit a BytesWarning when
compare strings and bytes. After fixing this the size will grow even
more. And there may be other issues with this code which will be found
months later. Later, when we rewrite the optimizer or change the
structure of AST we will need to update this code too, with possibility
to introduce new bugs.

On other hand, the benefit of this optimization is exact zero. It does
not make code faster, because it optimizes the code not used in real
programs.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LOIK6ZX43QHQSNWWUSE6YE3O5YBVGJOM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Optimizing literal comparisons and contains

[Serhiy Storchaka]

29.11.21 14:32, Mark Shannon пише:
> Excluding  1 < 2 seems inconsistent.

It is not excluded, it is just not included. There should be reasons for
adding any feature, and the benefit should exceed the cost.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/53Q5J2MHMQRIJL4KD5FSWRAQICDQXDUT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Optimizing literal comparisons and contains

[Serhiy Storchaka]

27.11.21 15:47, Jeremiah Vivian пише:
> Many operations involving two literals are optimized (to a certain level). So 
> it sort of surprises me that literal comparisons are not optimized and 
> literal contains only convert the right operand to a constant if possible. 
> I'd like to implement optimizations for these especially for the literal 
> contains. There is a TODO in the ast optimizer for literal comparisons as 
> well, and that's another reason I would like to have these added.

Only these operations were optimized which were necessary or very
useful. The Python parser produces only sign-less numbers, so it was
necessary to optimize unary minus to make -5 as fast as 5. Binary plus
and minus are necessary to make complex constants. "**" and "<<" are
often used in expressions for limits, masks and flags (5*10**6, 2**32-1,
1<<12). There is a benefit of computing such constants at compile time.
"*" and "/" can be used to create self-documenting constats too
(24*60*60, 1/3). And finally, indexing of bytes objects gets an ASCII
code of the character (b'a'[0]), it is useful too.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/73G2WZDTKG57JKKUVOCFOIDEL3D75W4O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Optimizing literal comparisons and contains

[Serhiy Storchaka]

28.11.21 17:13, Skip Montanaro пише:
>> That is not entirely true: 
>> https://github.com/python/cpython/pull/29639#issuecomment-974146979
> 
> The only places I've seen "if 0:" or "if False:" in live code was for
> debugging. Optimizing that hardly seems necessary. In any case, the
> original comment was about comparisons of two constants. I suppose
> sweeping up all of that into a constant expression folding/elimination
> step performed on the AST and/or during peephole optimization would
> cover both cases.

"if 0:" and "if False:" is already optimized by the compiler. The OP
proposes to optimize "2 < 1", and I cannot imagine any use case for this.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZJPTMHWPTPWC6Q4ZKWJB5CY6LURQ75PM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Do we need to remove everything that's deprecated?

[Serhiy Storchaka]

12.11.21 12:55, Petr Viktorin пише:
>   AttributeError: '[...]Tests' object has no attribute 'failUnless'
> (bpo-45162)

This one caused me troubles more then one time. It is so easy to make a
typo and write assertEquals instead of assertEqual or assertRaisesRegexp
instead of assertRaisesRegex. Tests are passed, and warnings are
ignored. Then I run tests with -Werror to test new warnings and get a
lot of unrelated failures because of PRs merged at last half-year. I am
very glad that these long time ago deprecated aliases are finally removed.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BUFGDRDX336TODI5I6JAA5SB42WYFFRX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Serhiy Storchaka]

10.11.21 01:47, Bob Fang пише:
> 1) Some mainstream language support them out of box: C++ for example
> have set/map which are sorted by the key order, and Java has TreeMap
> which is internally a Red-black tree.

>From my C++ and Java experience, hashtable-based containers are much
more useful than tree-based containers. They are more compact and fast.
In most cases the only reason of using sorted container is supporting
deterministic iteration order, but often it is enough to sort data only
for output. It is easy to do with the sorted() builtin in Python.

We need some dict implementatin in Python, it is essential part of the
language used to implement modules, classes and objects. And
hashtable-based implementation is good for this. There is no such need
of tree-based implementation. It is not needed in Python core, and is
not needed for most users. So it is good to keep it in third-party
libraries. Maintaining it in the stdlib has a cost and the benefit/cost
value would be low.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LG6GNFGRI4OSBEWF6KTIWLI65UXWMTIM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: pre-PEP: Unicode Security Considerations for Python

[Serhiy Storchaka]

03.11.21 15:14, Stephen J. Turnbull пише:
> So the only
> time that wouldn't be true is if escape sequences are allowed to
> represent characters.  I believe unicode_escape is the only codec
> that does.

Also raw_unicode_escape and utf_7. And maybe punycode or idna, I am not
sure.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XRORKXTTV55YOSMP7Z7MAL4AG2UQRXHK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Preventing Unicode-related gotchas (Was: pre-PEP: Unicode Security Considerations for Python)

[Serhiy Storchaka]

03.11.21 14:31, Petr Viktorin пише:
> For example: should the parser emit a lightweight audit event if it
> finds a non-ASCII identifier? (See below for why ASCII is special.)
> Or for encoding declarations?

There are audit events for import and compile. You can also register
import hooks if you want more fanny preprocessing than just
unicode-encoding. I do not think we need to add more specific audit
events, they were not designed for this.

And I think it is too late to detect suspicious code at the time of its
execution. It should be detected before adding that code to the code
base (review tools, pre-commit hooks).

> I don't think this would actually ban Cyrillic/Greek.
> (My suggestion is not vanilla confusables detection; it might require
> careful reading: "should there be a [linter] warning when an identifier
> looks like ASCII but isn't?")

Yes, but it should be optional and configurable and not be the part of
the Python compiler. This is not our business as Python core developers.

> I am not a native speaker, but I did try a bit to find an actual
> ASCII-like word in a language that uses Cyrillic. I didn't succeed; I
> think they might be very rare.

With simple script I have found 62 words common between English and
Ukrainian: гасу/racy, горе/rope, рима/puma, міх/mix, etc. But there are
much more English and Ukrainian words which contains only letters which
can be confused with letters from other script. And identifiers can
contains abbreviations and shortening, they are not all can be found in
dictionaries.

> Even if there was such a word -- or a one-letter abbreviation used as a
> variable name -- it would be confusing to use. Removing the possibility
> of confusion could *help* Cyrillic users. (I can't speak for them; this
> is just a brainstorming idea.)

I never used non-Latin identifiers in Python, but I guess that where
they are used (in schools?) there is a mix of English and non-English
identifiers, and identifiers consisting of parts of English and
non-English words without even an underscore between them. I know
because in other languages they just use inconsistent transliteration.
Emitting any warning by default is a discrimination of non-English
users. It would be better to not add support of non-ASCII identifiers at
first place.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XHHXRWGKTDTZIYGS6AB3DKEVFH5D6BHV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: pre-PEP: Unicode Security Considerations for Python

[Serhiy Storchaka]

03.11.21 12:36, Petr Viktorin пише:
> On 03. 11. 21 2:58, Kyle Stanley wrote:
>> I'd suggest both: briefer, easier to read write up for average user in
>> docs, more details/semantics in informational PEP. Thanks for working
>> on this, Petr!
> 
> Well, this is the brief write-up :)
> Maybe it would work better if the  info was integrated into the relevant
> parts of the docs, rather than be a separate HOWTO.
> 
> I went with an informational PEP because it's quicker to publish.

What is the supposed target audience of this document? If it is core
Python developers only, then PEP is the right place to publish it. But I
think that it rather describes potential issues in arbitrary Python
project, and as such, it will be more accessible as a part of the Python
documentation (as a HOW-TO article perhaps). AFAIK all other
informational PEPs are about developing Python, not developing in Python
(even if they are (mis)used (e.g. PEP 8) outside their scope).

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TM7EU4QHHXTJMXGQT2EJRKZYZ764HNAD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: pre-PEP: Unicode Security Considerations for Python

[Serhiy Storchaka]

03.11.21 11:01, Stephen J. Turnbull пише:
>  And of
> course UTF-16 is incompatible in that sense, although I don't know if
> anybody actually saves Python code in UTF-16.

CPython does not currently support UTF-16 for source files.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KN4MPLKSRKQOJM2DUFQNO4UGGOJN5YNU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Preventing Unicode-related gotchas (Was: pre-PEP: Unicode Security Considerations for Python)

[Serhiy Storchaka]

02.11.21 18:49, Jim J. Jewett пише:
> If escape sequences were also allowed in comments (or at least in strings 
> within comments), this would make sense.  I don't like banning them 
> otherwise, since odd characters are often a good reason to need a comment, 
> but it is definitely a "mention, not use" situation.

If you mean backslash-escaped sequences like \u, there is no reason
to ban them in comments. Unlike to Java they do not have special meaning
outside of string literals. But if you mean terminal control sequences
(which change color or move cursor) they should not be allowed in comments.

> At the time, we considered it, and we also considered a narrower restriction 
> on using multiple scripts in the same identifier, or at least the same 
> identifier portion (so it was OK if separated by _).

I implemented this restrictions in one of my projects. The character set
was limited, and even this did not solve all issues with homoglyphs.

I think that we should not introduce such arbitrary limitations at the
parser level and left it to linters.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2TT3VX4D4FMRSEOV4O2ICYTC7VC5M2J4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Preventing Unicode-related gotchas (Was: pre-PEP: Unicode Security Considerations for Python)

[Serhiy Storchaka]

02.11.21 16:16, Petr Viktorin пише:
> As for \0, can we ban all ASCII & C1 control characters except
> whitespace? I see no place for them in source code.

All control characters except CR, LF, TAB and FF are banned outside
comments and string literals. I think it is worth to ban them in
comments and string literals too. In string literals you can use
backslash-escape sequences, and comments should be human readable, there
are no reason to include control characters in them. There is a
precedence of emitting warnings for some superficial escapes in strings.


> For homoglyphs/confusables, should there be a SyntaxWarning when an
> identifier looks like ASCII but isn't?

It would virtually ban Cyrillic. There is a lot of Cyrillic letters
which look like Latin letters, and there are complete words written in
Cyrillic which by accident look like other words written in Latin.

It is a work for linters, which can have many options for configuring
acceptable scripts, use spelling dictionaries and dictionaries of
homoglyphs, etc.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DN24FK3A2DSO4HBGEDGJXERSAUYK6VK6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: pre-PEP: Unicode Security Considerations for Python

[Serhiy Storchaka]

This is excellent!

01.11.21 14:17, Petr Viktorin пише:
>> CPython treats the control character NUL (``\0``) as end of input,
>> but many editors simply skip it, possibly showing code that Python
>> will not
>> run as a regular part of a file.

It is an implementation detail and we will get rid of it. It only
happens when you read the Python script from a file. If you import it as
a module or run with runpy, the NUL character is an error.

>> Some characters can be used to hide/overwrite other characters when
>> source is
>> listed in common terminals:
>>
>> * BS (``\b``, Backspace) moves the cursor back, so the character after it
>>   will overwrite the character before.
>> * CR (``\r``, carriage return) moves the cursor to the start of line,
>>   subsequent characters overwrite the start of the line.
>> * DEL (``\x7F``) commonly initiates escape codes which allow arbitrary
>>   control of the terminal.

ESC (``\x1B``) starts many control sequences.

``\1A`` means the end of the text file on Windows. Some programs (for
example "type") ignore the rest of the file.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CBI7ME3YUAVVH5B6LSC745GJSVUIZJHO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Type Syntax

[Serhiy Storchaka]

12.10.21 00:03, Guido van Rossum пише:
> But if I see
> 
>     def Comparison(a: T, b: T) -> Literal[-1, 0, 1]:
>     ...
> 
> my first thought is that it's a comparison function that someone hasn't
> finished writing yet, not a function type -- since if it did have at
> least one line of code in the body, it *would* be that.

I agree. There are also problems with variables substitution (indexing),
union (operator "|") and introspection. We would need to add
__getitem__, __or__, __args__, __parameters__ etc to all functions.
There may be also conflicts with NewType which returns a callable.

But can it be expressed via Protocol?

class Comparison(Protocol, Generic[T]):
def __call__(self, /, a: T, b: T) -> Literal[-1, 0, 1]: ...

Then we can just add a decorator which turns a function into class:

@functype
def Comparison(a: T, b: T) -> Literal[-1, 0, 1]:
...

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ENIIWHG7DX3MTBAHOP6ST43J6MHVQKII/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: What is __int__ still useful for?

[Serhiy Storchaka]

14.10.21 12:24, Eryk Sun пише:
> Maybe an alternate constructor could be added -- such as
> int.from_number() -- which would be restricted to calling __int__(),
> __index__(), and __trunc__().

See thread "More alternate constructors for builtin type" on Python-ideas:
https://mail.python.org/archives/list/python-id...@python.org/thread/5JKQMIC6EUVCD7IBWMRHY7DRTTNSBOWG/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NU3774YDVCIUH44C7RZXCSSVRVYSLCUI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: What is __int__ still useful for?

[Serhiy Storchaka]

13.10.21 20:10, Antoine Pitrou пише:
> It used to be that defining __int__ allowed an object to be accepted as
> an integer from various functions, internal and third-party, thanks to
> being implicitly called by e.g. PyLong_AsLong.
> 
> Today, and since bpo-37999, this is no longer the case.  It seems that
> __int__ has now become a strict equivalent to __trunc__.  Of course,
> user code can still look up and call the __int__ method explicitly (or
> look up the nb_int slot, in C), but that's a bit of anti-pattern.
> 
> Is there a point in having three special methods __index__, __int__ and
> __trunc__, if two are them are practically interchangeable?

Today __int__ allows the object be explicitly converted to int. It is
defined for example in UUID and IPv4Address. We do not want them to be
converted to int implicitly or be valid argument of math.trunc().

__trunc__ adds support of the type in math.trunc(). There is no
requirement that it should return an int. GMP numbers can return GMP
integers and NumPy arrays can return NumPy arrays (I do not know whether
they do). It is similar to __floor__, __ceil__ and __round__.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IW4TTWZ2LGZOYUST2CA4X5HXM4XNWA72/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Packing a long list of numbers into memory

[Serhiy Storchaka]

11.10.21 01:35, MRAB пише:
> Maybe what's needed is to add, say, '*' to the format string to indicate
> that multiple values should come from an iterable, e.g.:
> 
>     struct.pack_into(f'{len(nums)}*Q', buf, 0, nums)
> 
> in this case len(nums) from the nums argument.

See https://www.python.org/dev/peps/pep-3118/.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DLG3FLOADSMM2SLRIKDCS3EVW3XAZ3HM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Why doesn't peephole optimise away operations with fast locals?

[Serhiy Storchaka]

10.10.21 23:38, Guido van Rossum пише:
> What's exasperating to me about this whole discussion is that nobody has
> shown any reason why this kind of code would be occurring in user code.
> STORE_FAST x LOAD_FAST x seems that it must come from a user writing
> 
>     x = x

No, it comes from pretty common code:

   for x in a:
   y = x**2

   with open() as f:
   for x in f:
   ...

   x = get():
   if x:
   ...

But there is nothing to optimize here. This sequence is already fast. It
can be replaced with "DUP_TOP; STORE_FAST x", but it would not be
faster. Even if we introduce a new combined opcode, it would not be
significantly faster, and it can slow down the eval loop.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EUYIABPM3QKP34ZG3YTHWDRYGQL2EY4P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Packing a long list of numbers into memory

[Serhiy Storchaka]

10.10.21 19:05, Patrick Reader пише:
> This still isn't a completely direct write - you're still creating an array 
> in between which is then copied into shm, whereas struct.pack_into writes 
> directly into the shared memory with no intermediate buffer.
> 
> And unfortunately you can't do
> 
> shm.buf[l_offset:r_offset].cast('Q')[:] = nums
> 
> where nums is a plain Python iterable; it has to already be an array, so it 
> still requires a copy.

Yes, I tried this too. Maybe we will add a support of arbitrary
iterables. There may be problems with this because we don't know the
length of the iterable and can't guarantee that all items are
convertible without converting them and saving results in a temporary array.

If you do not want to spend additional memory, iterate nums and set
items one by one. Or iterate and set them by chunks

   target = shm.buf[l_offset:r_offset].cast('Q')
   for i in range(0, len(nums), chunksize):
   j = min(i + chunksize, len(nums))
   target[i:j] = array.array('Q', nums[i:j])

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4IYA25GXYMUQ5CHORD7YYYVUUUDCGWAF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Packing a long list of numbers into memory

[Serhiy Storchaka]

10.10.21 18:18, Facundo Batista пише:
> You mean `array` from the `array` module? The only way I see using it
> is like the following:
> 
 shm = shared_memory.SharedMemory(create=True, size=total_size)
 a = array.array('Q', nums)
 shm.buf[l_offset:r_offset] = a.tobytes()
> 
> But I don't like it because of the `tobytes` call, which will produce
> a huge bytearray only to insert it in the shared memory buffer.
> 
> That's why I liked `pack_into`, because it will write directly into
> the memory view.
> 
> Or I'm missing something?

   shm.buf[l_offset:r_offset].cast('Q')[:] = a

or

   shm.buf[l_offset:r_offset] = memoryview(a).cast('B')


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/A2XKNKG4C55VKII6VOXRR3XEP6VHIU43/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Packing a long list of numbers into memory

[Serhiy Storchaka]

10.10.21 17:19, Facundo Batista пише:
> I have a long list of nums (several millions), ended up doing the following:
> 
> struct.pack_into(f'{len(nums)}Q', buf, 0, *nums)

Why not use array('Q', nums)?

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OWZXJRTQPZZKBUM4KMZWDWTBFELRAI46/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Type Syntax

[Serhiy Storchaka]

07.10.21 19:41, S Pradeep Kumar пише:
> 1. Syntax to replace Callable
> 
> After a lot of discussion, there is strong consensus in typing-sig about
> adding syntax to replace Callable. So, the above example would be
> written as:
> ```python
> def print_purchases(
>     user: User,
>     formatter: (PurchaseRecord, List[AuthPermission]) -> FormattedItem,
> ) -> None:
> 
>     <...>
>     output = formatter(record, permissions)
>     print(output)
> ```

How could you replace Callable[..., int] and Callable[Concatenate[str,
P], int] ?

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BJHVYIMVXYRGRTR77TNX77HEQKPZ2A4Y/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should I care what -3.14 // inf produces?

[Serhiy Storchaka]

30.09.21 10:07, Jeff Allen пише:
 from decimal import Decimal
 Decimal('-3.14') // Decimal('Infinity')
> Decimal('-0')

You do not need an infinity to see the difference.

>>> Decimal('3.14') % Decimal('2')
Decimal('1.14')
>>> Decimal('3.14') % Decimal('-2')
Decimal('1.14')
>>> Decimal('-3.14') % Decimal('2')
Decimal('-1.14')
>>> Decimal('-3.14') % Decimal('-2')
Decimal('-1.14')

Decimals use a different rule than integers and floats: the modulus has
the same sign as the dividend. It was discussed using this rule for
floats (perhaps there is even FAQ or HOWTO for this), there are
advantages of using it for floats (the result is more accurate). But the
current rule (the modulus has the same sign as the divisor) is much much
more convenient for integers, and having different rules for integers
and floats is a source of bugs.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YBIDA44TP4EA3L3OP6TQUHA6UWA6EJR2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Changing exception text in micro releases

[Serhiy Storchaka]

24.09.21 16:51, Eric V. Smith пише:
> This is clearly an improvement. My question is: is it okay to backport
> this to 3.9 and 3.10? I think we have a rule that it's okay to change
> the exception text, but I'm not sure how that applies to micro releases
> (3.9.x, 3.10.1).

I am sure that it will break someone's tests. Since the current error
message is not wrong, it is not a bug fix.

If you want it very much you can discuss backporting to 3.10.1 with
Pablo. Few people will upgrade immediately after releasing 3.10.0, and
for others this change will be a part of upgrading to 3.10. But 3.9 is
mature enough for now.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OSL4OGXHBCJ27K4V7HPFZIYIQACEY2EA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: f-strings in the grammar

[Serhiy Storchaka]

20.09.21 14:18, Pablo Galindo Salgado пише:
> * The parser will likely have "\n" characters and backslashes in
> f-strings expressions, which currently is impossible:

What about characters "\x7b", "\x7d", "\x5c", etc?

What about newlines in single quotes? Currently this works:

f'''{1 +
2}'''

But this does not:

f'{1 +
2}'

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IJEJ5UVVKHEH6QGXZ3LONZPAITNMBULL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] deprecated-removed

[Serhiy Storchaka]

I always thought that the "deprecated" directive is used if the term of
removing the deprecated feature is not set yet, and the
"deprecated-removed" directive is used if it is set. After removing the
feature both directives are replaced with the "versionchanged" directive
which only specifies the version of removing, not version of deprecation.

But after reading the devguide [1] I am no longer so confident. The
devguide clearly describes it as specifying version at which the feature
"is" removed, not "will be" removed. Maybe I always used this directive
incorrectly? Or its meaning was changed with time? Or I incorrectly
understand the devguide wording? Or devguide is incorrect?

Before 3.8 I only seen "deprecated-removed" with future removing
version. The first use "deprecated-removed" for feature which is already
deleted happened in 3.8 (maybe the directive was just not updated), and
there are now many uses for features removed in asyncio in 3.10.

What was the initial meaning of "deprecated-removed", how it should be
used now, and is the devguide correct?

[1]
https://devguide.python.org/documenting/?highlight=deprecated-removed#paragraph-level-markup


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SWBM2N4EFPVQFP4TX6Q33L5OK2WPBFRU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [Python-ideas] Re: open functions in dbm submodule need to support path-like object

[Serhiy Storchaka]

08.09.21 08:19, David Mertz, Ph.D. пише:
> I attempted to do this today, as my first actual contribution to CPython
> itself.  I think the prior attempt went down a wrong path, which is why
> neither PR could actually pass tests.
> 
> I've been looking at `posixmodule.c` for comparison, specifically.

The code in posixmodule.c is a bad example, because it is too general
and supports many options. It gives the patch as char* and wchat_t* (on
Windows), supports file descriptors and None, and format error messages
for functions supporting multiple types. But if you only need a path as
char*, you can just use PyUnicode_FSConverter().

There is an existing PR for this issue. It looks correct in general, but
I left some comments.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LQGU6DM5ZSDPCAXKLEII6YIF4HQI52NG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Discrepancy between what aiter() and `async for` requires on purpose?

[Serhiy Storchaka]

29.08.21 23:16, Brett Cannon пише:
> If you look at
> https://github.com/python/cpython/blob/b11a951f16f0603d98de24fee5c023df83ea552c/Python/ceval.c#L2409-L2451
> 
> you will see that `async for` requires that the iterator returned from
> `__aiter__` define `__anext__`. But if you look at aiter() which uses
> PyObject_GetAiter() from
> https://github.com/python/cpython/blob/f0a6fde8827d5d4f7a1c741ab1a8b206b66ffd57/Objects/abstract.c#L2741-L2759
> 
> and PyAiter_Check() from
> https://github.com/python/cpython/blob/f0a6fde8827d5d4f7a1c741ab1a8b206b66ffd57/Objects/abstract.c#L2769-L2778
> 
> you will notice that aiter() requires `__anext__` *and* `__aiter__` on
> the async iterator that gets returned from __aiter__.
> 
> Now the docs for aiter() at
> https://docs.python.org/3.10/library/functions.html#aiter
>  points out
> that the async iterator is expected to define both methods as does the
> glossary definition for "asynchronous iterator"
> (https://docs.python.org/3.8/glossary.html#term-asynchronous-iterator
> ).
> 
> So my question is whether the discrepancy between what `async for`
> expects and what `aiter()` expects on purpose?
> https://bugs.python.org/issue31861 
> was the issue for creating aiter() and I didn't notice a discussion of
> this difference. The key reason I'm asking is this does cause a
> deviation compared to the relationship between `for` and `iter()` (which
> does not require `__iter__` to be defined on the iterator, although
> collections.abc.Iterator does). It also makes the glossary definition
> being linked from
> https://docs.python.org/3.10/reference/compound_stmts.html#the-async-for-statement
> 
> incorrect.

PyIter_Check() only checks existence of __next__, not __iter__ (perhaps
for performance reasons).

I just ported changes from PyPy in SQLite tests
(https://github.com/python/cpython/pull/28021) because a test class with
__next__ but without __iter__ passed tests on CPython but failed on PyPy.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VWZQPBE5SLKA5IDOGSOJYFWUDOQ7HJKS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Making code object APIs unstable

[Serhiy Storchaka]

13.08.21 20:24, Guido van Rossum пише:
> If these weren't part of the stable ABI, I'd choose (E). But because
> they are, I think only (A) or (B) are our options. The problem with (C)
> is that if there's code that links to them but doesn't call them (except
> in some corner case that the user can avoid), the code won't link even
> though it would work fine. The problem with (D) is that if it *is*
> called by code expecting the old signature it will segfault. I'm not
> keen on (A) since it can cause broken code objects when used to copy a
> code object with some modified metadata (e.g. a different filename),
> since there's no way to pass the exception table (and several other
> fields, but the exception table is an integral part of the code now).

I agree that (A) and (B) are only options if we preserve binary
compatibility. For practical reasons I prefer (B).

We can make (A) working if add the exception table to the end of the
bytecode array and the endline/column tables to the end of the lineno
table. It would allow to re-construct the code object with some simple
changes (like filename or replace some constants). Creating the code
object from zero is version-specific in any case, because bytecode is
changed in every version, and semantic of some fields can be changed too
(e.g. support of negative offsets in the lineno table). But it would
complicate the code object structure and the code that works with it in
long term.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/B4OGG4EQUB4LPRLQRBZ2RBX4DVQJILST/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should we try to get a complete draft of What's New by b1?

[Serhiy Storchaka]

11.08.21 21:35, Brett Cannon пише:
> So my question is whether we want to push to be more diligent about
> updating What's New by b1 so people can provide feedback during the
> betas beyond just reporting breaking changes?

I think that What's New should be updated as soon as possible,
immediately after the addition of the feature. I started contributing to
Python after reading What's New for one of early alphas of 3.3 (and the
questioned feature was changed later). There were other examples, when
features were advertised in alphas, but were changed after getting a
feedback before betas.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RQQ2QEV3OKCMAJGHLB52SKXLAIL7BB34/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] types.Union or types.UnionType?

[Serhiy Storchaka]

In 3.10 the union type (the type of the result of the | operator for
types) was added (https://www.python.org/dev/peps/pep-0604/). It is
exposed as types.Union. There are differences between typing.Union and
types.Union:

* typing.Union is indexable, types.Union is not.
* types.Union is a class, typing.Union is not.

types.Union corresponds to private class typing._UnionGenericAlias, not
typing.Union. It is confusing that typing.Union and types.Union have the
same name but are so different. Note also that most classes in the types
module have the "Type" suffix: FunctionType, MethodType, ModuleType,
etc. I think that it would be better to rename types.Union to
types.UnionType.

The name of types.Union is the part of already accepted PEP 604, so we
need to hear opinions of authors, sponsor and BDFL-delegate of the PEP.

I did not participate in the discussion about PEP 604 so I do not know
if there are arguments in favor of types.Union over types.UnionType.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RHRF7Q25GQ3EIEUJHW72YLQNMCUDLWRW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Delayed evaluation of f-strings?

[Serhiy Storchaka]

24.06.21 12:37, Eric Nieuwland пише:
> In a recent discussion with a colleague we wondered if it would be
> possible to postpone the evaluation of an f-string so we could use it
> like a regular string and .format() or ‘%’.

You can just use lambda:

delayed_fstring = lambda: f"The current name is {name}"
print(delayed_fstring())

If you want to get rid of () when convert to string, you can use a wrapper:

class LazyStr:
def __init__(self, str_func):
self._str_func = str_func
def __str__(self):
return self._str_func()

delayed_fstring = LazyStr(lambda: f"The current name is {name}")
print(delayed_fstring)

but it works only if str() is directly or indirectly called for the
value. You cannot pass it open() as a file name, because open() expects
only str, bytes, integer of path-like object. Most builtin functions
will not and could not work with LazyStr.

As for evaluating variables in different scope as in your example, it
just does not work in Python. Python uses static binding, not dynamic
binding. First than introducing "delayed f-strings" you need to
introduce dynamic binding in normal functions.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IG6HEHUXY3K4WJ2PYIG47II7ESOIAQIK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Making PY_SSIZE_T_CLEAN not mandatory.

[Serhiy Storchaka]

07.06.21 08:41, Hai Shi пише:
> There have an another question. There have many C API defined under 
> PY_SSIZE_T_CLEAN, for example: _PyArg_Parse_SizeT().
> Could we remove or merge them after making PY_SSIZE_T_CLEAN not mandatory?

We should support binary compatibility to some degree, so there should
be several steps:

* Make macro PyArg_Parse an alias of _PyArg_Parse_SizeT. Keep function
PyArg_Parse.
* Make function PyArg_Parse always raising an exception.
* Remove function PyArg_Parse.
* [Optionally] Now we can re-add function PyArg_Parse as an alias of
_PyArg_Parse_SizeT and remove macro PyArg_Parse.
* [Optionally in 4.0 or 5.0] Remove _PyArg_Parse_SizeT.

But we can squish several last steps in 4.0 which do not need to support
binary compatibility with 3.x.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/J2D5VBSOQTZOK42AUM5AHPPBA45VNJK4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Making PY_SSIZE_T_CLEAN not mandatory.

[Serhiy Storchaka]

07.06.21 06:05, Inada Naoki пише:
> Since Python 3.8, PyArg_Parse*() APIs and Py_BuildValue() APIs emitted
> DeprecationWarning when
> '#' format is used without PY_SSIZE_T_CLEAN defined.
> In Python 3.10, they raise a RuntimeError, not a warning. Extension
> modules can not use '#' format with int.
> 
> So how about making PY_SSIZE_T_CLEAN not mandatory in Python 3.11?
> Extension modules can use '#' format with ssize_t, without
> PY_SSIZE_T_CLEAN defined.
> 
> Or should we wait one more version?

Many users still use 3.6 or 3.7. Jumping from 3.7 to 3.11 could break
extensions in bad way (crash, truncated data, leaked sensitive
information, execution of arbitrary code). Also, deprecation warnings in
3.8 and 3.9 can be easily ignored.

I propose to wait until both of conditions became true:

* 3.7 no longer maintained
* 3.10 reaches security-only mode.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UXJAG63XCVRUC4ENDF7V2Q6FLLBFWM2H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: declare "unstable APIs"

[Serhiy Storchaka]

06.06.21 06:48, Guido van Rossum пише:
> On Fri, Jun 4, 2021 at 6:15 AM Victor Stinner  > wrote:
> If possible, I would prefer to make PyThreadState, PyCodeObject and
> other structures opaque, and only go through getter and setter
> functions ;-) PyCode_New() is another problem :-/ The PEP 570 first
> changed it to add a new parameter. It broke Cython and other projects.
> The change was reverted, and PyCode_NewWithPosOnlyArgs() was added.
> The lesson is that it's possible to change PyCodeObject without
> breaking PyCode_New() (which handles the "backward compatibility" for
> you).
> 
> 
> I'm afraid that won't always be possible though. At some point there
> just may not be a valid meaning for the original PyCode_New() call. It
> was easy in the case of positional arguments (by default don't have any)
> but it may not always be that simple, and we shouldn't make guarantees here.

We have already reached this limit. In 3.11 the code object needs a
table of exception handlers. Only simplest code which do not contain any
"try" or "with" can now be created with old PyCode_New() and
PyCode_NewWithPosOnlyArgs().

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/G5Y5DD4T3ZSJ4DYWFZH4TFVNHJHBN5K2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: declare "unstable APIs"

[Serhiy Storchaka]

04.06.21 12:08, Petr Viktorin пише:
> It is used, and I started the work :)
> See e.g.
> https://docs.python.org/3.10/c-api/sequence.html#c.PySequence_Concat

Great news! Several core developers (including me) tried to solve this
problem, but a large amount of work stopped us at an early stage. Glad
there is a progress.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/44XXQIFBIUNTYTDH7XSN3XKBZ6SLYM5K/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: declare "unstable APIs"

[Serhiy Storchaka]

03.06.21 20:10, Guido van Rossum пише:
> This is not a complete thought yet, but it occurred to me that while we
> have deprecated APIs (which will eventually go away), and provisional
> APIs (which must mature a little before they're declared stable), and
> stable APIs (which everyone can rely on), it might be good to also have
> something like *unstable* APIs, which will continually change without
> ever going away or stabilizing. Examples would be the ast module (since
> the AST structure changes every time the grammar changes) and anything
> to do with code objects and bytecode (since we sometimes think of better
> ways to execute Python).
> 
> So maybe the docs should grow a standard way of saying "this is an
> unstable API"?

There is already a way to specify the stable ABI (see
Doc/tools/extensions/c_annotations.py). But unfortunately this feature
is not is not used in the documentation. It needs just an amount of work
to do this, and nobody did this.

After marking all stable ABI we can extend this feature to support
halftones: provisional API, unstable API for Cython, etc.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KAOTXYRGZMTNB4TYJBQTQAEEG2PJZBDI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: In what tense should the changelog be written?

[Serhiy Storchaka]

30.04.21 05:57, Larry Hastings пише:
> When one writes one's "blurb" for the changelog, in what tense should it
> be?

See the previous discussion on this topic:
https://mail.python.org/archives/list/python-dev@python.org/thread/C342Y73LALVFLI2ACFB3SH6ZDT2YTGPC/
.

I always use "fixed", "removed", "added" when describe changes following
the rule formulated by Oleg Broytman:

> I use past tense to describe what I did on the code, and present
> simple to describe what the new code does when running.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6SXYOA73FGFRP4HND6UM6SXGT23ZH7PS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: str() vs format(): trivia question

[Serhiy Storchaka]

20.04.21 17:56, Ethan Furman пише:
> urllib.urlencode currently uses `str()` on its non-bytes objects before
> encoding the result.  This causes a compatibility break when integer
> module constants are converted to IntEnum, as `str(IntEnum.MEMBER)` no
> longer returns the integer representation; however, `format()` does
> still return the integer representation.
> 
> The fix is to add a separate branch to check if the argument is an Enum,
> and use the value if so -- but it got me wondering: in general, are
> there differences between calling str() vs calling format() on Python
> objects?

format() without format specifier and str() should return the same value
in general, otherwise it will confuse users. But str() for enum should
in general return a symbolic name, not the attached value which is an
implementation detail. This is the purpose of enums.

I think that __format__ should return the same as __str__ by default.
This can break some user code, but custom __str__ can break it as well.
Some breakage is inevitable when we convert some constants in the stdlib
to enums. If user want to get an integer representation of an enum
member, it should use IntEnum.MEMBER.value or int(IntEnum.MEMBER).

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TGWLHZA7F6OW35IAGTQBLTAURRQMACCP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: str() vs format(): trivia question

[Serhiy Storchaka]

20.04.21 22:01, Guido van Rossum пише:
> So to be clear, that one user wants f"{Color.RED}" to return "1" and not
> " Color.RED" (or  something like that).

The user should write f"{int(Color.RED)}" or f"{Color.RED.value}".

I have also an idea to support of additional conversion characters, so
the use could write f"{Color.RED!i}". Opened a discussion for this on
Python-ideas.
https://mail.python.org/archives/list/python-id...@python.org/thread/3AALXB6A7EN6UY635MF5O2SFHZVFA5NM/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UB4PDYSREAEUXAEUVCIZQA4G5KJPFEVD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Can we change python -W option and PYTHONWARNINGS to use a regex for the message?

[Serhiy Storchaka]

16.04.21 16:07, Victor Stinner пише:
> I propose to change the -W command line option and the PYTHONWARNINGS
> environment variable to use the message as a regular expression in
> Python 3.10. Or does anyone have a reason to keep the current behavior
> as it is?
> 
> I created https://bugs.python.org/issue43862 for this change.

It is known issue, but changing it is would break compatibility. The
warning message can contain characters which have special meaning in
regular expressions (e.g. "(" and "?"). Event if it can be parsed as a
regular expression, it can stop to work.

It would be more safe to add some flag which control whether the message
is a pattern or literal. For example, if it starts with "/" it is a
pattern (no warning message or module name starts with "/").

Module name also should support regular expressions.

Note also that specifying message or module name in the -W option or the
PYTHONWARNINGS environment variable causes importing the re module at
the start of the interpreter, and it nontrivially increases the starting
time.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UCFRLK3HCVLJ25VM3OL45XVXC4KMFLTR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: How about using modern C++ in development of CPython ?

[Serhiy Storchaka]

24.03.21 11:54, redrad...@gmail.com пише:
> What about of using modern C++ in developing CPython ?
> 
> With C++ we can get the following benefits:
> 1) Readability - less code, most code is hidden by abstraction without losing 
> performance
> 2) Maintainability - no code duplication in favor of using reusable classes
> 3) RAII - Resource Acquisition Is Initialization, predictable allocation and 
> free resources
> ...

The drawbacks:

1. You can only use C++ internally. All API should be in pure C. It
reduces possible benefits.
2. If you use C++, even internally, you have to link with libstdc++ and
use C++ compatible linker.
3. C++ is much much more complex than pure C. It drastically reduces the
number of available contributors and maintainers.

Also note that the C code it already written. The C++ code has yet to be
written, with chances to introduce new bugs during rewriting and
distracting resources from other tasks.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4X4QEE7IFR5WGKBSHMSNVVYWCS3VX4JW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: 0xfor 3. Parser bug?

[Serhiy Storchaka]

14.04.21 13:22, Stefano Borini пише:
> This was just posted on SO
> 
> https://stackoverflow.com/questions/67083039/why-does-python-return-15-for-0xfor-x-in-1-2-3
> 
> I can reproduce it with a simpler example
> 
 0xfor 3
> 15
> 
> Is it a bug in the parser, or working as intended? It's not only for
> hex. This works to
> 
 3or 50
> 3
> 
> 
> 

It is already discussed in thread >
https://mail.python.org/archives/list/python-dev@python.org/thread/D2WPCITHG2LBQAP7DBTC6CY26WQUBAKP/

It does not contradict specification, but looks pretty confusing, so we
will likely change specification and implementation to prevent confusion.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QODMSOOHZFGL7HL4VDAKZMZ23RPYTIQO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Boundaries between numbers and identifiers

[Serhiy Storchaka]

26.04.18 21:37, Serhiy Storchaka пише:
> In Python 2.5 `0or[]` was accepted by the Python parser. It became an
> error in 2.6 because "0o" became recognizing as an incomplete octal
> number. `1or[]` still is accepted.
> 
> On other hand, `1if 2else 3` is accepted despites the fact that "2e" can
> be recognized as an incomplete floating point number. In this case the
> tokenizer pushes "e" back and returns "2".
> 
> Shouldn't it do the same with "0o"? It is possible to make `0or[]` be
> parseable again. Python implementation is able to tokenize this example:
> 
> $ echo '0or[]' | ./python -m tokenize
> 1,0-1,1:    NUMBER '0'
> 1,1-1,3:    NAME   'or'
> 1,3-1,4:    OP '['
> 1,4-1,5:    OP ']'
> 1,5-1,6:    NEWLINE    '\n'
> 2,0-2,0:    ENDMARKER  ''
> 
> On other hand, all these examples look weird. There is an assymmetry:
> `1or 2` is a valid syntax, but `1 or2` is not. It is hard to recognize
> visually the boundary between a number and the following identifier or
> keyword, especially if numbers can contain letters ("b", "e", "j", "o",
> "x") and underscores, and identifiers can contain digits. On both sides
> of the boundary can be letters, digits, and underscores.
> 
> I propose to change the Python syntax by adding a requirement that there
> should be a whitespace or delimiter between a numeric literal and the
> following keyword.
> 

New example was found recently (see https://bugs.python.org/issue43833).

 >>> [0x1for x in (1,2)]
 [31]

It is parsed as [0x1f or x in (1,2)] instead of [0x1 for x in (1,2)].

Since this code is clearly ambiguous, it makes more sense to emit a
SyntaxWarning if there is no space between number and identifier.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XYAEYTON7Q2XNUYYQ65DHSHE2JRHNYYX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NamedTemporaryFile and context managers

[Serhiy Storchaka]

08.04.21 23:31, Ethan Furman пише:
> In issue14243 [1] there are two issues being tracked:
> 
> - the difference in opening shared files between posix and Windows
> - the behavior of closing the underlying file in the middle of
>   NamedTemporaryFile's context management
> 
> I'd like to address and get feedback on the context management issue.
> 
> ```python
> from tempfile import NamedTemporaryFile
> 
> with NamedTemporaryFile() as fp:
>     fp.write(b'some data')
>     fp = open(fp.name())
>     data = fp.read()
> 
> assert data == 'some_data'
> ```


These issues are usually solved by using TemporaryDirectory:

with TemporaryDirectory() as td:
filename = os.path.join(td, 'tempfile')
with open(filename, 'wb') as fp:
fp.write(b'some data')
with open(filename, 'rb') as fp:
data = fp.read()

What if make NamedTemporaryFile a wrapper around TemporaryDirectory and
always create a new temporary directory for file?

@contextmanager
def NamedTemporaryFile():
with TemporaryDirectory() as td:
filename = os.path.join(td, 'tempfile')
with open(filename, 'wb') as fp:
yield fp

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2H2PIMPZ3VE2JDBUA7WVX25ULIC3C4SM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: When we remove 'U' mode of open()?

[Serhiy Storchaka]

07.04.21 19:13, Victor Stinner пише:
> Hi Inada-san,
> 
> I'm +0 on removing again the flag, but I would prefer to not endorse
> the responsibility. I am already responsible for enough incompatible
> changes in Python 3.10 :-D
> 
> Some context on this "U" open mode. The flag is accepted by many
> functions opening files. It is deprecated (emit DeprecationWarning)
> for 9 years (Python 3.3, 2012).

It was silently deprecated before 3.3 (perhaps it was no-op since 3.0).

I added DeprecationWarning with intention to remove this option in all
functions accepting it. The only non-trivial support of the "U" mode was
left in ZipFile.open(), and it was broken since beginning.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BEECCPDJUUE7MMLVD4GUCOQBDMLGP7ZE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Where and how can I contribute?

[Serhiy Storchaka]

26.03.21 20:37, Terry Reedy пише:
> On 3/26/2021 6:29 AM, Marco Sulla wrote:
>> I would contribute to the project in my spare time. Can someone point
>> me to some easy task? I know C and the Python C API a little.
> 
> Review existing PRs.  In some cases (ask), convert existing patches
> posted on bpo issues to PRs.

But before doing this ask if the original author want to create a PR.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QXWRSA5CWSELQSU36DGYFQUIQSJVHBHU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Non-monotonically increasing line numbers in dis.findlinestarts() output

[Serhiy Storchaka]

18.03.21 03:39, Victor Stinner пише:
> I'm happy to see that Python 3.10 now also implements faster bytecode
> which rely on this change ;-)

It was used long time ago before 3.10. For example:

a[i] = \
f()

  2   0 LOAD_NAME0 (f)
  2 CALL_FUNCTION0

  1   4 LOAD_NAME1 (a)
  6 LOAD_NAME2 (i)
  8 STORE_SUBSCR
 10 LOAD_CONST   0 (None)
 12 RETURN_VALUE

x = [
1,
2,
]

  2   0 LOAD_CONST   0 (1)

  3   2 LOAD_CONST   1 (2)

  1   4 BUILD_LIST   2
  6 STORE_NAME   0 (x)
  8 LOAD_CONST   2 (None)
 10 RETURN_VALUE

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/L24E77UPUVGEZPOXYVIVI76GGJARW6K6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [Release management] schedule for renaming the default branch

[Serhiy Storchaka]

10.03.21 16:06, Pablo Galindo Salgado пише:
> # What you need to do?
> 
> You just need to update your local clone after the branch name changes.
> From the local clone of the repository on a computer,
> run the following commands to update the name of the default branch.
> 
> $ git branch -m master main
> $ git fetch origin
> $ git branch -u origin/main main
> 
> Apart from that, you should update any local script or command that uses
> the name "master" to use the name "main".

I have above 200 feature branches in my local repository. Will renaming
the master branch cause any problems?

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/G2LLBPJMIBD2DOA7AVTYYA77PGLGYLUE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Signature discrepancies between documentation and implementation

[Serhiy Storchaka]

09.02.21 12:22, Erlend Aasland пише:
> What's the recommended approach with issues like 
> https://bugs.python.org/issue43094? Change the docs or the implementation? I 
> did a quick search on bpo, but could not find similar past issues.

If the documentation and the C implemented function contradict about
parameter name, we are free to treat the parameter as positional-only.
User cannot pass the argument as keyword because the documented name
does not work, and the real name is not exposed to the user.

In this case, there are two similar functions, create_function() and
create_aggregate(). Both are documented as having parameter
"num_params", but real names are different: "narg" and "n_arg". It would
be confusing to have different spelling of the same parameter in similar
functions. I am sure that it would be difficult to remember how is it
spelled in every case. Also, in Python terminology, the semantic of this
name is the number of parameters, not the number of argument.

So I think that in this particular case the chance of breaking some code
is insignificant, but possible long-term harm of exposing bad parameter
names may be significant.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JE2E5RJZQPMXHTWIU3NA74YDMEZHGYUK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: What's up with assignment expression and tuples?

[Serhiy Storchaka]

05.02.21 09:51, Paul Sokolovsky пише:
> a0 = 0
> b0 = 10
> while ((a1, b1) := phi([a0, a2], [b0, b2]))[0] < 5:
> a2 = a1 + 1
> b2 = b1 + 1

Such code quickly becomes unreadable. Especially if in real code
function has additional arguments and names are longer that 2-3 characters.

The following code is not much larger but more clear and extensible:

a0 = 0
b0 = 10
while True:
a1, b1 := phi([a0, a2], [b0, b2]))
if b1 >= 5:
break
a2 = a1 + 1
b2 = b1 + 1
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6YILI7IW3HYPMZSPH3DJYWMNHJKZ2CXN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Bumping minimum Sphinx version to 3.2 for cpython 3.10?

[Serhiy Storchaka]

13.01.21 01:28, Victor Stinner пише:
> I looked at Sphinx and Python versions of Debian, Ubuntu and Fedora:
> https://bugs.python.org/issue42843#msg384963
> 
> In my list, there is only Debian Buster (stable) which doesn't have
> Sphinx 3 yet. It uses Python 3.7 and so would not be affected by
> Python 3.8 changes.

Also Ubuntu. Ubuntu 20.04 has only Sphinx 1.8.5, but Python 3.8 and 3.9.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4AUTIFRA4ANJMYCFTUCY2RVUIVI3LRJC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Bumping minimum Sphinx version to 3.2 for cpython 3.10?

[Serhiy Storchaka]

12.01.21 22:38, Julien Palard via Python-Dev пише:
> During the development of cpython 3.10, Sphinx was bumped to 3.2.1.
> 
> Problem is Sphinx 3 have some incompatibilities with Sphinx 2, some that 
> we could work around, some are bit harder, so we may need to bump 
> `needs_sphinx = '3.2'` (currently it is 1.8).

Sphinx version in the current Ubuntu LTS (20.04) is 1.8.5. Would not it
cause problems with builting documentation on Ubuntu?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EJWTFPHZUX522RNCTIGAAOHWD23VD7NQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

[Serhiy Storchaka]

12.01.21 03:21, Larry Hastings пише:
> I forgot about __slots__!  Yup, it's optional, and you can even delete
> it, though after the class is defined I'm not sure how much difference
> that makes.

It affects pickling if __slotnames__ is not set yet. The latter is set
when you pickle or copy an instance the first time.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YR3U2DT4ZDXOO5MXQFPVV3A53B4TQPHW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: why include cpython/*.h, Need this macro ? # error "this header file must not be included directly"

[Serhiy Storchaka]

10.01.21 09:53, junyixie пише:
> #ifndef Py_LIMITED_API
> #  define Py_CPYTHON_FILEOBJECT_H
> #  include  "cpython/fileobject.h"
> #  undef Py_CPYTHON_FILEOBJECT_H
> #endif
> 
> cpython/fileobject.h
> ```
> #ifndef Py_CPYTHON_FILEOBJECT_H
> #  error "this header file must not be included directly"
> #endif
> ```
> 
> why not use  #ifndef #define 
> cpython/fileobject.h
> #ifndef Py_CPYTHON_FILEOBJECT_H
> #define Py_CPYTHON_FILEOBJECT_H
> 
> #endif /* Py_CPYTHON_FILEOBJECT_H */

Because it will not produce compile-time error when you include that
header files directly.

The division of these declarations on few files is a deep implementation
detail. Header file "cpython/fileobject.h" is not a part of API and we
do not want users to use them directly and then fail because in next
feature (or even bugfix) release some declarations have been moved to
other files.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UCKJUHCR4JLSPJKKQBKSUSRTNDBNARVI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Where is the SQLite module maintainer

[Serhiy Storchaka]

27.12.20 22:20, Erlend Aasland пише:
> Who can help me review code that touches the sqlite3 module code base?
> 
> 
> I've received a lot of constructive reviews from Victor Stinner, Dong-he
> Na and Pablo Galindo on my past sqlite3 PR's; thank you so much! I still
> feel uncomfortable requesting their review, as none of them are sqlite3
> module maintainers.

I fixed many bugs in sqlite3 and did reviews, so I have some experience,
although I am not an expert. I payed less attention to it since Berker
Peksag took its maintenance, so my attention was not needed. But seems
he is not very active now (I hope it is temporary). You can request my
review. My reaction may be not instant because last weeks I am very busy
with curses and tkinter.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DYA4XLJMSAEHDU4YQEG5QEU5SGRT6HBC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Why does "except Ex as x" not restore the previous value of x?

[Serhiy Storchaka]

17.11.20 11:55, Mark Shannon пише:
> I'm wondering why
> ```
> x = "value"
> try:
>     1/0
> except Exception as x:
>     pass
> ```
> 
> does not restore "value" to x after
> the `except` block.
> 
> There doesn't seem to be an explanation for this behavior in the docs or
> PEPs, that I can find.
> Nor does there seem to be any good technical reason for doing it this way.

Others already said that it is because "except" does not create a new
scope. But why it does not create a new scope? Because it is a design of
Python. In general, since in Python local variables do not have
declarations, inner scopes do not work. For example:

y = 1
if x:
y = 2

If "if" create a new scope, "y" after "if" would be restored to 1.

But what if make an exception for "except" and make a special rule for it?

First, "Special cases aren't special enough to break the rules". Second,
how would you use it? This feature would only encourage to write
hard-to-read and errorprone code. This error is so common, that in some
other programming languages which support inner scopes shadowing local
variable causes compiler warning or even syntax error. So this feature
is a misfeature.

There is also historical reason. In Python 2 the variable was not
deleted after leaving the except block. Now it is deleted, and if the
code uses the variable after this it would raise a NameError. It is a
clear indication of program error. With your proposition it would have
some other value, and program error would not be caught so easily.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LMMNJLNWGCWVMDSFFZHSPYT4RFIEO3ZR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: When to remove BytesWarning?

[Serhiy Storchaka]

11.11.20 15:05, John Hagen пише:
> If I recall, it was str(bytes) warning that flagged in a few places and was 
> missing a .decode() call or similar.
> 
> It seems like the bytes== warnings could be implemented in a type checker 
> such as mypy, if it doesn't
> already do this. Assuming you have correct type coverage/inference on your 
> project, you could
> potentially catch this at static analysis time rather than runtime.

There were several bugs like sep=='/' (where sep can be bytes) in the
stdlib. These cases were not covered by tests, so they were fixed only
in 3.3 or even later. I hope all such bugs are already fixed, but I
cannot guarantee.

And there were bugs with str(bytes) too.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OQNQIEKS3UTPIHROL66OTQIRG3NX67D6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Who is target reader of tutorial?

[Serhiy Storchaka]

05.11.20 11:12, Inada Naoki пише:
> Since "How To" guide is not organized well, it is very tempting to
> write all details in tutorial.
> I have seen may pull requests which "improve" tutorial by describe
> more details and make
> the tutorial more perfect.
> 
> But adding more and more information into tutorial makes it slower to
> learn Python by reading tutorial.
> 10+ years ago, Python is far less popular in Japan and reading
> tutorial is the best way to learn Python to me.
> 
> But now Python is popular and there are many free/paid good books and
> tutorials on the Web.
> Some of them would be more new user friendly than official tutorial.
> Then, should official Python tutorial become detailed guide to Python?
> Or should we keep new user learning Python as targeted reader?

I think the tutorial should be targeted to new Python users, which
already know one or more programming languages. It should not be an
introduction into programming, nor a detailed guide. I myself learned
Python by just reading tutorial. Contained enough information to start
coding and to understand some fundamental differences of Python from
other programming languages, but not too heavy. Additional details I
learned from the library reference. For advanced topics there are "How
to" and the language reference.

At that time (Python 2.2) it was the best tutorial that I read. Other
language tutorials was either too large and formal, or spent several
chapters explaining what is bits and loops.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5RPOGC2RHTFIJO5KUTRV24NENRNYXQBF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Drop Solaris, OpenSolaris, Illumos and OpenIndiana support in Python

[Serhiy Storchaka]

29.10.20 23:43, Victor Stinner пише:
> * Current best effort support (no change): changes only happen if a
> core dev volunteers to review and merge a change written by a
> contributor.

It is my preference.

Several years ago I tested and fixed Python on OpenIndiana in virtual
machine, but I was not enough motivated and the code gradually degrades.

> * Schedule the removal in 2 Python releases (Python 3.12) and start to
> announce that Solaris support is going to be removed
> 
> * Remove the Solaris code right now (my proposition): Solaris code
> will have to be maintained outside the official Python code base, as
> "downstream patches"

It would make life of Illumos and OpenIndiana developers much harder,
that can be seen hostile to open source community. It would make the
code of CPython more rigid, virtually Linux-only with Windows and MacOS
patches, and as a side effect can make harder porting it to other
platforms. This is not great.

If you want to remove platform specific code from CPython source, do you
consider the idea of removing support of MacOS and maintain it as
"downstream patches"?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WKCE33CIVYFQCSHBUI3W23DT5PKE3ZRF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Macro for logging

[Serhiy Storchaka]

21.10.20 15:30, Antoine Pitrou пише:
> On Wed, 21 Oct 2020 14:19:37 +0200
> Marco Sulla  wrote:
>> If not already present, do you think it's useful to add a macro that does
>> something like
>>
>> # ifdef Py_DEBUG
>> fprintf(stderr, "%s\n", message);
>> # endif
> 
> In general, you want to do this on a per-component basis, so each C
> source file tends to redefine its own macros if it needs to.

I concur with Antoine.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GKP52NSQEE4I4JVC7LVEO2LQISCVXFO3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: When to remove BytesWarning?

[Serhiy Storchaka]

24.10.20 06:19, Inada Naoki пише:
> To avoid BytesWarning, the compiler needs to do some hack when they
> need to store bytes and str constants in one dict or set.
> BytesWarning has maintenance costs. It is not huge, but significant.
> 
> When can we remove it? My idea is:
> 
> 3.10: Deprecate the -b option.
> 3.11: Make the -b option no-op. Bytes warning never emits.
> 3.12: Remove the -b option.
> 
> BytesWarning will be deprecated in the document, but not to be removed.
> Users who want to use the -b option during 2->3 conversion need to use
> Python ~3.10 for a while.

I agree that it should be removed, and that BytesWarning should be kept
(maybe we will reuse it for other purposes in future).

But I do not see how deprecating it before removing could help. Using it
with -We will no longer work, and without -We it will just add a noise.

We can just make -b a no-op at any moment and remove it few versions
later. Or maybe first make it no-op, then deprecate, then remove. But it
looks too much.

-b is still usable in 3.9, so it can be removed not earlier than EOL of
3.9. Users that use it should be able to use it with all maintained
Python versions if it makes sense with at least one of them.

3.x: Make the -b option no-op. Bytes warning never emits.
3.x+4: Remove the -b option.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6LBAYEQSWUEFKNR6LMJ35OAF2YZXAVWE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: When to remove BytesWarning?

[Serhiy Storchaka]

26.10.20 10:10, Inada Naoki пише:
> Of course, there are some runtime costs too.
> 
> https://github.com/python/cpython/blob/fb5db7ec58624cab0797b4050735be865d380823/Modules/_functoolsmodule.c#L802
> https://github.com/python/cpython/blob/fb5db7ec58624cab0797b4050735be865d380823/Objects/codeobject.c#L724
> (maybe more, but I'm not sure)

It will not help much in these cases because we still need to
distinguish 1 from True and 1.0 and -0.0 from 0.0.

But if keys only can be str or bytes, we pay additional cost. An example
is the re cache.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/S4B7KIIEB5XRPE4WKDXMSN424DJGNGBC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Pickle for C extension?

[Serhiy Storchaka]

19.10.20 20:39, Marco Sulla пише:
> TL;DR Is it possible to use C code to implement the (un)pickling of an
> type written in a C extension, as it was written in _pickle.c?
> 
> Long explaining: I'm trying to create a C extension for frozendict. For
> simplicity, first I wrote it in CPython, then I started to move it in a
> C extension. It seems to work, but I have to move the code I wrote in
> _pickle.c and pickle.py in the C extension. Is it possible, or I have to
> create a slower `__reduce_ex__` method that simply converts it to dict?
> 
> This is, for example, the C code for pickling frozendict in _pickle.c:
> https://github.com/Marco-Sulla/cpython/blob/41a640a947c36007e56bbc28f362c261110d2001/Modules/_pickle.c#L3370

For adding new opcode to the pickle format you need a new pickle version
and a PEP.

Implement __reduce_ex__, it is the way to support pickling of new types
(as it was for sets, bytearrays, and many extension types).
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WLBLDASK25BWANWS347U4HIOBA763IEW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Remove module's __version__ attributes in the stdlib

[Serhiy Storchaka]

14.10.20 20:56, Brett Cannon пише:
> I think if the project is not maintained externally and thus synced into
> the stdlib we can drop the attributes.

I have found only one exception. decimal.__version__ refers to the
version of the external specification which was not changed since 2009.
I think it should be kept, although it might be better to use different
name for it (like "spec_version").

I do not know about any current projects maintained externally and
synced into the stdlib. simplejson and ElementTree are too different now
from the stdlib versions. Some features flow in both directions, but
selectively on case by case basis, not as full sync. External argparse
is outdated now.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MIEMWWC5W2WKV25WTARXACQOIUBUUSLS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Remove module's __version__ attributes in the stdlib

[Serhiy Storchaka]

15.10.20 14:59, Victor Stinner пише:
> If the __version__ variable is used, I suggest to start with a
> deprecation period using a module __getattr__(): emit
> DeprecationWarning, and only remove these variables in 2 Python
> releases (PEP 387).

This is a good idea, I though about it. But it seems that there are no
usages the __version__ variable in top 4K pypi packages.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LBHVJIXGG3U5FPOYYOTS6AG3KPSLHBER/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Remove module's __version__ attributes in the stdlib

[Serhiy Storchaka]

14.10.20 23:25, Batuhan Taskaya пише:
> I've indexed a vast majority of the files from top 4K pypi packages to
> this system, and here are the results about __version__ usage on
> argparse, cgi, csv, decimal, imaplib, ipaddress, optparse, pickle,
> platform, re, smtpd, socketserver, tabnanny (result of an quick grep)
> 
> 
> rawdata/clean/argparse/setup.py
> 
> |argparse.__version__|

If it refers to the third-party argparse module, which uses
argparse.__version__ in its setup.py, it is __version__ of that
third-party module, not the one from the stdlib.

> rawdata/pypi/junitparser-1.4.1/bin/junitparser
> 
> |argparse.__version__|

argparse.__version__ is used for displaying the version of the
junitparser script. Of course the version of argparse (1.1 in the
stdlib) does not have any relation with the version of junitparser
(currently 1.4.1), so this is purely a misuse.


> rawdata/pypi/interpret_community-0.15.1/interpret_community/mlflow/mlflow.py
> 
> 
> |pickle.__version__|
> 
> The pickle in the last example looks like a result of import cloudpickle
> as pickle, so we are safe to eliminate that.

So it seems that there is only one usage of __version__ from the stdlib
modules, and that that one is a bug. Reported.

It seems pretty safe to just remove __version__ variables.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GRJTXCRUDEUQZE2IYSBKFJYKGS7AXZXA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Remove module's __version__ attributes in the stdlib

[Serhiy Storchaka]

14.10.20 23:25, Batuhan Taskaya пише:
> I've indexed a vast majority of the files from top 4K pypi packages to
> this system, and here are the results about __version__ usage on
> argparse, cgi, csv, decimal, imaplib, ipaddress, optparse, pickle,
> platform, re, smtpd, socketserver, tabnanny (result of an quick grep)
> 
> 
> rawdata/clean/argparse/setup.py
> 
> |argparse.__version__|
> 
> rawdata/pypi/junitparser-1.4.1/bin/junitparser
> 
> |argparse.__version__|
> 
> rawdata/pypi/interpret_community-0.15.1/interpret_community/mlflow/mlflow.py

As for argparse, it was perhaps the last third-party module added to the
stdlib without changing name and significant rewriting. It was added in
Python 2.7/3.2, and older Python versions are not maintained for long
time. There is a third-party module argparse on PyPI for older Python
versions, its version 1.4 is higher that the version in the stdlib
(1.1), but I think that the stdlib version has more features. The
version of the module is just not informative.

> |pickle.__version__|
> 
> The pickle in the last example looks like a result of import cloudpickle
> as pickle, so we are safe to eliminate that.
> 
> Here is the query if you want to try by yourself on different
> parameters:
> https://search.tree.science/?query=Attribute%28Name%28%27argparse%27%7C%27cgi%27%7C%27csv%27%7C%27decimal%27%7C%27imaplib%27%7C%27ipaddress%27%7C%27optparse%27%7C%27platform%27%7C%27pickle%27%7C%27re%27%7C%27smtpd%27%7C%27socketserver%27%7C%27tabnanny%27%29%2C+%22__version__%22%29

Thank you Batuhan. It will help to decide what to do with __version__
attributes: keep them ,upgrade to sys.version or sys.version_info, remove.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XAHSO7AUX5J7MDXFDXUAHYAAWF7WH4JZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Remove module's __version__ attributes in the stdlib

[Serhiy Storchaka]

Some module attributes in the stdlib have attribute __version__. It
makes sense if the module is developed independently from Python, but
after inclusion in the stdlib it no longer have separate releases which
should be identified by version. New changes goes into module usually
without changing the value of __version__. Different versions of the
module for different Python version can have different features but the
same __version__.

I propose to remove __version__ in all stdlib modules. Are there any
exceptions?

Also, what do you think about other meta attributes like __author__,
__credits__, __email__, __copyright__, __about__, __date__?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KBU4EU2JULXSMUZULD5HJJWCGOMN52MK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [python-committers] Resignation from Stefan Krah

[Serhiy Storchaka]

09.10.20 16:48, Antoine Pitrou пише:
> Also, we now have an unmaintained module
> (_decimal) requiring specific competence.

It is not so bad. Due to high quality of the module there were not much
changes in it in recent years, and many of them were cosmetic. We also
have other Decimal experts.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CCITUQVXSEQELH3ZAQTQTHTWGBVVSASD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Do we still need a support of non-type objects in issubclass()?

[Serhiy Storchaka]

04.10.20 01:06, Guido van Rossum пише:
> On Sat, Oct 3, 2020 at 9:28 AM Serhiy Storchaka  <mailto:storch...@gmail.com>> wrote:
> The code of object.__dir__() is very old, it predates new-style classes,
> and currently it gathers names using different algorithm than used in
> object.__getattr__(), so object.__dir__() does not always return a list
> of names accepted by object.__getattr__().
> Would anything break if we changed `dir()` to use `__mro__` instead of
> `__bases__`? It would probably be simpler.

It is what I planned to play with. Also we can consider using
_PyObject_GetDictPtr() instead of resolving the __dict__ attribute and
Py_TYPE() instead of resolving the __class__ attribute.

> > I think all that hackery may predate (and may even have been an
> > inspiration for features of) new-style classes.
> I wonder whether it should pass with Python 2.
> I suppose you meant "pass" as in "die". I agree.

Do we need a deprecation period?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PGK7OYU2XAFUTDIYPLG6O5PJGRU5J2D6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Do we still need a support of non-type objects in issubclass()?

[Serhiy Storchaka]

03.10.20 18:15, Guido van Rossum пише:
> Is this the only place where a non-class object with __bases__ is
> accepted? Or do we have more such? I recall that long ago you could use
> certain non-class objects as base classes.

The only other place is object.__dir__(). It recursively iterates the
__class__.__bases__ chain, and __class__ can be a non-type object.

The code of object.__dir__() is very old, it predates new-style classes,
and currently it gathers names using different algorithm than used in
object.__getattr__(), so object.__dir__() does not always return a list
of names accepted by object.__getattr__().

> I think all that hackery may predate (and may even have been an
> inspiration for features of) new-style classes.

I wonder whether it should pass with Python 2.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GRSFMWDHH55KN4KZSDXSJLKWWYDF2FKE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Do we still need a support of non-type objects in issubclass()?

[Serhiy Storchaka]

03.10.20 12:45, Steven D'Aprano пише:
> On Sat, Oct 03, 2020 at 11:41:16AM +0300, Serhiy Storchaka wrote:
>> issubclass() accept types (extension types and new-style classes),
>> classic classes and arbitrary objects with the __bases__ attribute as
>> its arguments.
> 
> I didn't think classic classes were still possible in Python 3. How do 
> you get them?

They were originally in Python 2.2.2 code when the support of objects
with the __bases__ attribute was added. In Python 3 there are no classic
classes, but some code was written to support them.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LWQTXQUGHTDCD7PU2RVW3VOQFOPSBEHP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Do we still need a support of non-type objects in issubclass()?

[Serhiy Storchaka]

issubclass() accept types (extension types and new-style classes),
classic classes and arbitrary objects with the __bases__ attribute as
its arguments. The latter was added in issue464992 [1] to support the
Zope extension ExtensionClass.

I tested the current code of ExtensionClass [2], and seems it does not
need that special support. Tests are passed on modified master which
uses tp_bases instead of dynamical look up of __bases__,
ExtensionClass.ExtensionClass is a subclass of type and
ExtensionClass.Base is an instance of type, the code does not have the
__bases__ property and does not set the __bases__ attribute explicitly.

On other hand, the code in the Python core has a non-zero maintenance
cost. It was rewritten several times for 19 years, there is a gigantic
comment larger that the current code explaining it (and it is slightly
outdated), and there was several bugs related to it [3] [4].

Should we continue to support non-type objects with __bases__ in
issubclass()? If no, can we remove their support in 3.10 (it would fix a
crash in issue41909) or after passing a deprecation period?

[1] https://bugs.python.org/issue464992
[2] https://github.com/zopefoundation/ExtensionClass
[3] https://bugs.python.org/issue39382
[4] https://bugs.python.org/issue41909
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2S2FM2J44H2FQTZZOC7MINFVMZMQKCOP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Understanding why object defines rich comparison methods

[Serhiy Storchaka]

23.09.20 03:05, Greg Ewing пише:
> On 23/09/20 12:20 am, Steven D'Aprano wrote:
>> Presumably back when rich comparisons were added, the choice would have
>> been:
>>
>> - add one tp_richcompare slot to support all six methods; or
>>
>> - add six slots, one for each individual dunder
>>
>> in which case the first option wastes much less space.
> 
> I don't know the exact reasons, but it might also have been
> because the implementations of the six dunders are usually
> very closely related, so having just one function to implement
> at the C level is a lot easier for most types.
> 
> Also remember that before tp_richcompare existed there was
> only tp_compare, which also handled all the comparisons, so
> tp_richcompare was likely seen as a generalisation of that.

And before dunders __eq__, __lt__, etc there was a single dunder __cmp__.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WYF3DROV3X7TRVUSJAIIHCTFHPO5F7RB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Enum and the Standard Library

[Serhiy Storchaka]

22.09.20 16:57, Ethan Furman пише:
> On 9/22/20 12:11 AM, Serhiy Storchaka wrote:
>> The only exception is StrEnum -- overriding __str__ of str
>> subclass may be not safe. Some code will call str() implicitly, other
>> will read the string content of the object directly, and they will be
>> different.
> 
> Following up on that:
> 
>     >>> import enum
>     >>>
>     >>> class TestStr(enum.StrEnum):
>     ... One = '1'
>     ... Two = '2'
>     ... Three = '3'
>     ...
>     >>> isinstance(TestStr.One, str)
>     True
> 
>     >>> str(TestStr.One)
>     'TestStr.One'
> 
>     >>> TestStr.One == '1'
>     True
> 
> I agree, str.__str__ needs to be used in this case.

It is more interesting to compare '%s' % (TestStr.One,) and
'{}'.format(TestStr.One). Also str.upper(TestStr.One) and
int(TestStr.One) ignore __str__.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Z3UHCVPV6XQ4DP2QZAGNR4IYY565JG3D/
Code of Conduct: http://python.org/psf/codeofconduct/