[Python-Dev] Re: Having Sorted Containers in stdlib?

[Serhiy Storchaka]

10.11.21 01:47, Bob Fang пише:
> 1) Some mainstream language support them out of box: C++ for example
> have set/map which are sorted by the key order, and Java has TreeMap
> which is internally a Red-black tree.

>From my C++ and Java experience, hashtable-based containers are much
more useful than tree-based containers. They are more compact and fast.
In most cases the only reason of using sorted container is supporting
deterministic iteration order, but often it is enough to sort data only
for output. It is easy to do with the sorted() builtin in Python.

We need some dict implementatin in Python, it is essential part of the
language used to implement modules, classes and objects. And
hashtable-based implementation is good for this. There is no such need
of tree-based implementation. It is not needed in Python core, and is
not needed for most users. So it is good to keep it in third-party
libraries. Maintaining it in the stdlib has a cost and the benefit/cost
value would be low.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LG6GNFGRI4OSBEWF6KTIWLI65UXWMTIM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Paul Bryan]

On Wed, 2021-11-10 at 17:16 +1100, Steven D'Aprano wrote:
> On Tue, Nov 09, 2021 at 10:01:35PM -0800, Christopher Barker wrote:
> > Maybe a stupid question:
> > 
> > What are use cases for sorted dicts?
> > 
> > I don’t think I’ve ever needed one.
> 
> Good question :-)

It could be handy for deterministic iteration of its values, for
example to  allow serialized values to be easily compared, or to
generate and verify a signatures. It's edge case material; I wouldn't
think it's strong enough use case for inclusion in stdlib.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WHA7HC7U7VTLALPDP7BS5QLOFDG54Y4T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Chris Angelico]

On Wed, Nov 10, 2021 at 5:45 PM Steven D'Aprano  wrote:
>
> On Wed, Nov 10, 2021 at 05:11:33PM +1100, Chris Angelico wrote:
>
> > Nothing's technically new. You could make an inefficient sorted dict like 
> > this:
> >
> > class SortedDict(dict):
> > def __iter__(self): return iter(sorted(self.keys()))
>
> You would need more than that. You would want to ensure that the dict's
> repr shows it is sorted order. You would need popitem to pop items in
> the appropriate order. There may be other APIs that sorted dicts provide
> than merely sorting the keys on iteration.

Sure. That was a massively oversimplified one, to point out that the
question of whether this is "truly new" is fairly meaningless. A thing
can be extremely useful without actually being *new* per se.

> You don't actually need to implement this to see how repeatedly sorting
> the keys would give terrible performance for anything above small sets
> of data.

And yes. That is exactly the problem.

> > IMO this is a great tool to have on PyPI. Someone can start the naive
> > way, run into major performance problems, and then go "huh, maybe
> > someone else has already solved this problem". Doesn't need to be in
> > the stdlib for that.
>
> You may have missed that this thread has already discussed two mature,
> good quality sorted dict implementations that have been on PyPI for
> years:
>
> https://pypi.org/project/sortedcontainers/
>
> https://pypi.org/project/treap-python/
>
>
> Here are a few more:
>
> https://pypi.org/project/ruamel.ordereddict/
>
> https://github.com/surenkov/PySortedDict
>
> https://pypi.org/project/sorteddict/
>

I know it's on PyPI, in multiple forms. I'm saying that it's a great
tool to keep there, not in the stdlib.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QKFJJBYDAGHVFWSLSUSYS4WY7ECJ3ONQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Steven D'Aprano]

On Wed, Nov 10, 2021 at 05:11:33PM +1100, Chris Angelico wrote:

> Nothing's technically new. You could make an inefficient sorted dict like 
> this:
> 
> class SortedDict(dict):
> def __iter__(self): return iter(sorted(self.keys()))

You would need more than that. You would want to ensure that the dict's 
repr shows it is sorted order. You would need popitem to pop items in 
the appropriate order. There may be other APIs that sorted dicts provide 
than merely sorting the keys on iteration.

You don't actually need to implement this to see how repeatedly sorting 
the keys would give terrible performance for anything above small sets 
of data.

Here's Java's standard SortedMap:

https://docs.oracle.com/javase/7/docs/api/java/util/SortedMap.html

That should give you some idea of the interface a sorted dict would 
likely provide.


> IMO this is a great tool to have on PyPI. Someone can start the naive
> way, run into major performance problems, and then go "huh, maybe
> someone else has already solved this problem". Doesn't need to be in
> the stdlib for that.

You may have missed that this thread has already discussed two mature, 
good quality sorted dict implementations that have been on PyPI for 
years:

https://pypi.org/project/sortedcontainers/

https://pypi.org/project/treap-python/


Here are a few more:

https://pypi.org/project/ruamel.ordereddict/

https://github.com/surenkov/PySortedDict

https://pypi.org/project/sorteddict/


-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HPEXFUZR6Z6AHW4MHABWXT7UKCM6ELYZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Steven D'Aprano]

On Tue, Nov 09, 2021 at 10:01:35PM -0800, Christopher Barker wrote:
> Maybe a stupid question:
> 
> What are use cases for sorted dicts?
> 
> I don’t think I’ve ever needed one.

Good question :-)

 
> Also, I can’t quite tell from the discussion If a “sorted dict” implements
> something new, or is an internal data structure that gives better
> performance for particular use cases. I.e. is a sorted dict a Mapping?

All dicts are mappings :-) I would expect any kind of sorted dict to 
support the full mutable mapping interface.

Some mappings are not necessarily implemented as hash tables, as dict 
is. Some variety of tree is a common choice.

I expect that a sorted dict (however it is implemented) would be a 
mapping that preserves sorted order on insertions and deletions, rather 
than insertion order. (Or some arbitrary order.)

I haven't used one myself, but I would expect an API where you create 
a Sorted Dict (of whatever library or implementation you prefer), set an 
optional key function and direction (ascending or descending), then as 
you insert keys, the mapping keeps the keys in sort order.

Note that this is different from *sorting* a dict, which (if it were 
supported) has to be done explicitly. Between sorts, the dict would be 
capable of getting out of sorted order. The idea of a *sorted* dict is 
that the sort order is an invariant, rather than something that can come 
and go as you insert and delete items.


-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JCFQYOSD5NQ3GFGLRVW7QWISXQL5LHS5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Chris Angelico]

On Wed, Nov 10, 2021 at 5:03 PM Christopher Barker  wrote:
>
> Maybe a stupid question:
>
> What are use cases for sorted dicts?
>
> I don’t think I’ve ever needed one.

Me neither, tbh.

> Also, I can’t quite tell from the discussion If a “sorted dict” implements 
> something new, or is an internal data structure that gives better performance 
> for particular use cases. I.e. is a sorted dict a Mapping?
>

Nothing's technically new. You could make an inefficient sorted dict like this:

class SortedDict(dict):
def __iter__(self): return iter(sorted(self.keys()))

So in that sense, yes, it's better performance for the use-case of
needing the keys to be in order. It's not something I have ever really
needed - on the rare occasions when I need something like that, it's
usually no harder to maintain a separate sorted list or something.

IMO this is a great tool to have on PyPI. Someone can start the naive
way, run into major performance problems, and then go "huh, maybe
someone else has already solved this problem". Doesn't need to be in
the stdlib for that.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SL2UXHDTIUQQ6ARNLPBXZCEHVSVONRJH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Hasan Diwan]

A quick Google for "treap python github" yielded
https://github.com/TheAlgorithms/Python/blob/master/data_structures/binary_tree/treap.py
.

On Tue, 9 Nov 2021 at 21:49, Dan Stromberg  wrote:

>
> On Tue, Nov 9, 2021 at 9:00 PM Steven D'Aprano 
> wrote:
>
>> Sorting dicts has been discussed on the Python-Ideas mailing list, it is
>> too hard and expensive to justify for the limited use-cases for it. If
>> you want to sort a dict, you are best to sort the dict's keys, then
>> create a new dict. Or possibly use a dedicated Sorted Mapping type like
>> a red-black tree or similar.
>>
>
> There are several implementations of key-order sorted dicts available in
> Python, and more than a few of them are well tested.  I am the maintainer
> of one of them: https://pypi.org/project/treap/ which I ported from
> Java.  I also did a performance comparison among several of them a few
> years back.
>
> Red-black trees are popular, perhaps especially among Java developers, but
> I've never understood why.  They aren't the fastest.  Among traditional
> tree-based key-sorted dicts, treaps are frequently fastest.
>
> However, SortedDict is not uncommonly faster than treaps - I believe this
> is because SortedDict is very good at maximizing locality of reference.
> Traditional trees are almost always going to do an entire cache line hit
> for every node in large trees, even if those nodes are "next to each other"
> when sorted/traversed.  SortedDicts put keys that are nearly equal, in
> nearly the same part of memory - so multiple values can be retrieved with a
> single cache line hit.
>
> Sorting a dict's keys inside a loop tends to give O(n^2) algorithms, and
> sometimes even O(n^2 * logn).  This is not good.  A traditional tree should
> give O(nlogn) algorithms under similar circumstances, and although my gut
> is telling me SortedDict is similar the presentation linked below suggests
> a different (though still better than sorting in a loop) runtime for
> SortedDict.
>
> It's true that key-sorted dicts are not all that common.  Their most
> common use is probably for implementing finite caches - evictions can
> benefit from ordered keys.  However, we already have functools.lru_cache.
>
> Here's my most recent performance comparison:
> https://stromberg.dnsalias.org/~strombrg/sorted-dictionary-comparison/Datastructure%20comparison.pdf
>
> Here's a PyCon 2016 presentation about SortedContainers, that includes
> SortedDict:
> https://www.youtube.com/watch?v=7z2Ki44Vs4E
>
> I think it would make sense to include at least one key-sorted dict in
> CPython, and feel that SortedDict would not be a bad way to go.  Neither
> would treap.
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/VNK4VPGA4LJH7RMR4LCCACEG2WNDBBFO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
OpenPGP: https://hasan.d8u.us/openpgp.asc
If you wish to request my time, please do so using
*bit.ly/hd1AppointmentRequest
*.
Si vous voudrais faire connnaisance, allez a *bit.ly/hd1AppointmentRequest
*.

Sent
from my mobile device
Envoye de mon portable
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LQHSFACZUBNKIO33TJSVRMI7QBGUDS5D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Christopher Barker]

Maybe a stupid question:

What are use cases for sorted dicts?

I don’t think I’ve ever needed one.

Also, I can’t quite tell from the discussion If a “sorted dict” implements
something new, or is an internal data structure that gives better
performance for particular use cases. I.e. is a sorted dict a Mapping?

-CHB



On Tue, Nov 9, 2021 at 9:45 PM Dan Stromberg  wrote:

>
> On Tue, Nov 9, 2021 at 9:00 PM Steven D'Aprano 
> wrote:
>
>> Sorting dicts has been discussed on the Python-Ideas mailing list, it is
>> too hard and expensive to justify for the limited use-cases for it. If
>> you want to sort a dict, you are best to sort the dict's keys, then
>> create a new dict. Or possibly use a dedicated Sorted Mapping type like
>> a red-black tree or similar.
>>
>
> There are several implementations of key-order sorted dicts available in
> Python, and more than a few of them are well tested.  I am the maintainer
> of one of them: https://pypi.org/project/treap/ which I ported from
> Java.  I also did a performance comparison among several of them a few
> years back.
>
> Red-black trees are popular, perhaps especially among Java developers, but
> I've never understood why.  They aren't the fastest.  Among traditional
> tree-based key-sorted dicts, treaps are frequently fastest.
>
> However, SortedDict is not uncommonly faster than treaps - I believe this
> is because SortedDict is very good at maximizing locality of reference.
> Traditional trees are almost always going to do an entire cache line hit
> for every node in large trees, even if those nodes are "next to each other"
> when sorted/traversed.  SortedDicts put keys that are nearly equal, in
> nearly the same part of memory - so multiple values can be retrieved with a
> single cache line hit.
>
> Sorting a dict's keys inside a loop tends to give O(n^2) algorithms, and
> sometimes even O(n^2 * logn).  This is not good.  A traditional tree should
> give O(nlogn) algorithms under similar circumstances, and although my gut
> is telling me SortedDict is similar the presentation linked below suggests
> a different (though still better than sorting in a loop) runtime for
> SortedDict.
>
> It's true that key-sorted dicts are not all that common.  Their most
> common use is probably for implementing finite caches - evictions can
> benefit from ordered keys.  However, we already have functools.lru_cache.
>
> Here's my most recent performance comparison:
> https://stromberg.dnsalias.org/~strombrg/sorted-dictionary-comparison/Datastructure%20comparison.pdf
>
> Here's a PyCon 2016 presentation about SortedContainers, that includes
> SortedDict:
> https://www.youtube.com/watch?v=7z2Ki44Vs4E
>
> I think it would make sense to include at least one key-sorted dict in
> CPython, and feel that SortedDict would not be a bad way to go.  Neither
> would treap.
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/VNK4VPGA4LJH7RMR4LCCACEG2WNDBBFO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/U7FQIE5NDWY3AQI5QDMIJSIKVIQ3YUPC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Dan Stromberg]

On Tue, Nov 9, 2021 at 9:00 PM Steven D'Aprano  wrote:

> Sorting dicts has been discussed on the Python-Ideas mailing list, it is
> too hard and expensive to justify for the limited use-cases for it. If
> you want to sort a dict, you are best to sort the dict's keys, then
> create a new dict. Or possibly use a dedicated Sorted Mapping type like
> a red-black tree or similar.
>

There are several implementations of key-order sorted dicts available in
Python, and more than a few of them are well tested.  I am the maintainer
of one of them: https://pypi.org/project/treap/ which I ported from Java.
I also did a performance comparison among several of them a few years back.

Red-black trees are popular, perhaps especially among Java developers, but
I've never understood why.  They aren't the fastest.  Among traditional
tree-based key-sorted dicts, treaps are frequently fastest.

However, SortedDict is not uncommonly faster than treaps - I believe this
is because SortedDict is very good at maximizing locality of reference.
Traditional trees are almost always going to do an entire cache line hit
for every node in large trees, even if those nodes are "next to each other"
when sorted/traversed.  SortedDicts put keys that are nearly equal, in
nearly the same part of memory - so multiple values can be retrieved with a
single cache line hit.

Sorting a dict's keys inside a loop tends to give O(n^2) algorithms, and
sometimes even O(n^2 * logn).  This is not good.  A traditional tree should
give O(nlogn) algorithms under similar circumstances, and although my gut
is telling me SortedDict is similar the presentation linked below suggests
a different (though still better than sorting in a loop) runtime for
SortedDict.

It's true that key-sorted dicts are not all that common.  Their most common
use is probably for implementing finite caches - evictions can benefit from
ordered keys.  However, we already have functools.lru_cache.

Here's my most recent performance comparison:
https://stromberg.dnsalias.org/~strombrg/sorted-dictionary-comparison/Datastructure%20comparison.pdf

Here's a PyCon 2016 presentation about SortedContainers, that includes
SortedDict:
https://www.youtube.com/watch?v=7z2Ki44Vs4E

I think it would make sense to include at least one key-sorted dict in
CPython, and feel that SortedDict would not be a bad way to go.  Neither
would treap.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VNK4VPGA4LJH7RMR4LCCACEG2WNDBBFO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Steven D'Aprano]

Hi Bob and welcome,

Before we could even consider adding the sortedcontainers library to the 
standard library, we would need to hear from the maintainer(s) of the 
library that they agree to the move and would be able to continue 
maintaining the library under our release schedule and backwards 
compatibility guarantees.

Otherwise, you would need to find a core developer willing to 
re-implement the containers and maintain them.

I don't want to discourage you, but even if the maintainer is 
willing, we night not decide to add it. Every new feature, class and 
function adds to the weight of learning Python, and the cost of 
maintenance. We must balance that against the benefit, and only add 
features where the benefits are greater than the costs.

Our decision making is usually very conservative, because we have strong 
requirements for backwards-compatibility. Once we add something to the 
stdlib, we can't easily change our mind and remove it again. So we 
follow the Zen of Python:

>>> import this
The Zen of Python, by Tim Peters
[...]
Now is better than never.
Although never is often better than *right* now.


Have you looked at the Python PEPs? If you are serious about pushing 
this proposal, your first step should be to read PEP 1 and then browse 
through the collection of successful and unsuccessful PEPs:

https://www.python.org/dev/peps/pep-0001/

https://www.python.org/dev/peps/


-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VCM5PUEHCGRIYTPPRAZZUSEEJ3KW2DTN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Steven D'Aprano]

On Tue, Nov 09, 2021 at 04:23:50PM -0800, Hasan Diwan wrote:

> As of 3.7. dicts are sorted[1], but I'm unsure if the order can be
> overridden.


Dicts are not sorted, they are kept in insertion order.

>>> d = {3: 'a', 4: 'b', 1: 'c', 2: 'd'}
>>> d
{3: 'a', 4: 'b', 1: 'c', 2: 'd'}

See the docs:

https://docs.python.org/3/library/stdtypes.html#dict

although you have to scroll almost all the way to then end, just before 
the dict view objects, to see it documented.

Sorting dicts has been discussed on the Python-Ideas mailing list, it is 
too hard and expensive to justify for the limited use-cases for it. If 
you want to sort a dict, you are best to sort the dict's keys, then 
create a new dict. Or possibly use a dedicated Sorted Mapping type like 
a red-black tree or similar.


Hasan wrote:

> >>> print(cmp(Girls, Boys))
> Traceback (most recent call last):
>   File "", line 1, in 
> NameError: name 'cmp' is not defined

The 'cmp' built-in was removed in Python 3.0.

In any case, dicts don't support order comparisons.


-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AV5RVB37YR7TZ3ZURI4ZIHCS5Y2XV67Y/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Richard Damon]

On 11/9/21 3:24 PM, Terry Reedy wrote:

On 11/9/2021 1:52 PM, Sebastian Rittau wrote:

Am 09.11.21 um 19:26 schrieb Terry Reedy:
The signature of Sebastian's function with honest parameter names is 
foo(x_or_y, required_y=_NotGiven, /).  It is the 2nd argument, not 
the first, that is optional, as with range.  If required_y is not 
given, than x_or_y must be y, and x is given a default that is not 
part of the signature because it is explicitly bound when called. If 
required_y *is* given, then x_or_y can be x. 


Just to clarify: This proposal works differently than how range() 
works. foo(3) would be illegal as the required second parameter ("y") 
is missing.


No it is not.  If there is one required positional parameter and one 
supplies one positional argument, then that argument must be bound to 
that parameter name.


This can be solved by either supplying both "x" and "y", e.g. 
foo(None, 3), or by using a named parameter for "y": foo(y=3).


You are asking that 'x' be required if 'y' is passed by position but 
not if 'y' is passed by name.  This is contradictory.


Note that range does not allow passing by keyword and I specified the 
same for foo.  If you make 'y' keyword only, then there is no problem 
making 'x' optional.



the honest names are foo(x=None, y) in my proposal.


No, because you want 'x' is required or not depending on how 'y' is 
passed.




One big issue with that method is there are surprise traps.

Given that definition, and a example function defined as:

def fun(x=None, y):

then if in future we want to add a default to y, we can't as it would be 
a silent breaking change for a call like fun(2).



Now, if that call needed to be fun(y=2) then we avoided that break.

By first feeling is that the problems caused by skipping optional 
positional parameters is enough to not make it worth it.


There is also that fact that if we initially require the y only call to 
need to be a keyword parameter, but still be required and could be 
positional if both are provided, we could later allow it to be 
positional if that seems REALLY important to do, but it is much harder 
to take back a syntax that has been given.



--
Richard Damon

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XN6L7VXZFOLAX7BXQV3VIYWSMX3XKIHZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Hasan Diwan]

On Tue, 9 Nov 2021, 17:05 Bob Fang,  wrote:

> But that’s in insertion order, not in key order right? I think we need
> data structure that are key-ordered.
>
According to the tests, it seems to be key-ordered. It also appears that
the ordering cannot be changed (yet?). -- H
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KP2NMUHVOIGA5ZTZM7DXRCJZAN4C6EFS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Bob Fang]

But that’s in insertion order, not in key order right? I think we need data 
structure that are key-ordered. 

> On 10 Nov 2021, at 00:23, Hasan Diwan  wrote:
> 
> 
> On Tue, 9 Nov 2021 at 16:01, Bob Fang  > wrote:
> This is a modest proposal to consider having sorted containers 
> (http://www.grantjenks.com/docs/sortedcontainers/ 
> )
>  in standard library. I know that usually adding stuff to standard library 
> requires some strong arguments, so I will try my best to give my reasons here:
> 
> As of 3.7. dicts are sorted[1], but I'm unsure if the order can be 
> overridden. Indeed:
> 
> >>> Boys = {'Tim': 18,'Charlie':12,'Robert':25}
> Girls = {'Tiffany':22}  
> 
> >>> print(cmp(Girls, Boys))
> Traceback (most recent call last):
>   File "", line 1, in 
> NameError: name 'cmp' is not defined
> >>> Girls.__gt__(Boys)
> NotImplemented
> 
> -- H
> -- 
> OpenPGP: https://hasan.d8u.us/openpgp.asc 
> 
> If you wish to request my time, please do so using 
> bit.ly/hd1AppointmentRequest 
> .
> Si vous voudrais faire connnaisance, allez a bit.ly/hd1AppointmentRequest 
> .
> 
>  
> Sent
>  from my mobile device
> Envoye de mon portable
> 1. https://softwaremaniacs.org/blog/2020/02/05/dicts-ordered/ 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/RNXTH3JRUZ5WH33AKHVOSZF34FGNMS6S/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IQGAYFVN7556YUN3JE2EQA4TOSOVCNIE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: containment and the empty container

[Chris Angelico]

On Wed, Nov 10, 2021 at 11:44 AM Ethan Furman  wrote:
> I don't know if there's a formal name, but in my mind, if you have something 
> you don't have nothing.  If you have a
> barrel with nothing in it (not even air, just a hard vacuum) then saying you 
> have nothing in that barrel is a true
> statement; as soon as something is in that barrel, then saying you have 
> nothing in the barrel is a false statement.

You're defining nothingness as the absence of anything. "Is this
bitset empty?" is a test of nothingness that parallels what you're
asking. (I'm not sure what the truth value of these kinds of flagsets
is, but it would make sense for an empty flag set to be false.)

But asking if there is nothing in the barrel is a quirk of English.
Suppose we have a barrel with multiple colours of balls in it.

1) Do we have any red balls in the barrel?
2) Do we have any green balls in the barrel?
3) Do we have red and green balls in the barrel?

Clearly, testing for the combination red|green depends on both red and
green being there.

4) Do we have blue balls in the barrel?
5) Do we have red and blue balls in the barrel?

Same concept. Cool.

If you start thinking backwards, clearly going from "red|blue" to
"red" makes it easier for it to be true. And going from "red|blue" to
"blue" also makes it easier for it to be true. Removing a condition
from the set that's being tested will never change it from "yes we do"
to "no we don't". If we have some set of balls in the barrel, we must
logically have a subset of those balls.

The ultimate in flag removal is no flags at all. If going from
"red|blue" to "red" makes it easier to be true, and going from
"green|blue" to "green" does too, then logically, going from "blue" to
"nothing" should also. In other words, asking if we have all of, uhh,
no flags at all, should always be true.

It's a bit of a weird question to ask - kinda like "what is the
product of no numbers" (or if you prefer: "what is x to the zeroth
power") - but logic dictates the result.

(All we have to do is fix English so it's more logical.)

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CKODAMIJGEFYRB35CYCLMWUTVFMN7DIF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: containment and the empty container

[Ethan Furman]

On 11/8/21 3:09 PM, Steven D'Aprano wrote:
> On Mon, Nov 08, 2021 at 01:43:03PM -0800, Ethan Furman wrote:

>> SomeFlag.nothing in SomeFlag.something  <--  ???
>
> I don't think that consistency with other containers is particularly
> relevant here. More useful is consistency with other flag objects.

You mean from other programming languages?

> What's SomeFlag? I presume it is some sort of Enum, or bitset.

Yes.

> Presumably SomeFlag.nothing is equivalent to a bitset of all zeroes.

Yes.

> You might have:
>
>  something = 0b11010
>
>  0 in something # ???
>  1 in something # False
>  2 in something # True
>  4 in something # False
>  8 in something # True
>  16 in something # True
>  32 in something # False
>
> So much is obvious. But what about `3 in something`?
>
> If that is interpreted as an *all* operation, you get:
>
>  3 in something --> all(i in something for i in (1, 2)) # False
>
> but if it is an *any* operation:
>
>  3 in something --> any(i in something for i in (1, 2)) # True

It's an `all()` operation.

> *If* that is how you interpret your containment tests, that implies a
> natural interpretation for `nothing in something`. The vacuous truth of
> all is True, and of any is False:
>
>  0 in something --> all(i in something for i in ()) # True
>  0 in something --> any(i in something for i in ()) # False
>
> Vacuous truth is not the only possible interpretation. If you have some
> other obvious and natural interpretation of `0 in something` then you
> probably wouldn't be asking here, but if you did, you could follow that
> interpretation. With a good reason to violate the vacuous truth rules,
> it would only be a *little* surprising.

To rephrase Wikipedia [1]: a vacuously true statement doesn't really say 
anything.

I prefer my truths to actually say something. ;-)

I don't know if there's a formal name, but in my mind, if you have something you don't have nothing.  If you have a 
barrel with nothing in it (not even air, just a hard vacuum) then saying you have nothing in that barrel is a true 
statement; as soon as something is in that barrel, then saying you have nothing in the barrel is a false statement.


All of that is academic without a solid (and possibly irritating ;) use-case, so let's look at a different portion of 
the standard library -- opening files with `os.open()`:


Help on built-in function open in module posix:

open(path, flags, mode=511, *, dir_fd=None)
Open a file for low level IO.  Returns a file descriptor (integer).

A possible `flags` value (and presumably the default value) is `O_RDONLY` which is `0`.  I can imagine an implementation 
similar to (using integers):


if flags == 0:
stuff
if flags & O_TRUNC == O_TRUNC:
stuff
if flags & O_RDRW == O_RDRW:
stuff
# preliminary stuff done
return do_open_file(...)

If that were converted to use Flag, then the above would still work, but if 
written anew could become:

flags = OpenFlag(flags)# in case older code passes in an integer
# ignoring O_RDONLY for now
#
if O_TRUNC in flags:
stuff
if O_RDRW in flags:
stuff

Which is pretty straight-forward.  Circling back to O_RDONLY -- in the above 
snippet we either:

- need to remember that that particular flag has an actual value of 0 (and one 
of the big motivating
  factors behind Enum was not having to worry about the actual value) and use a 
different operator:

if not flags:
if flags == 0:  # only works because OpenFlag is an IntFlag for backwards 
compatibility
if flags.value == 0  # works for both Flag and IntFlag

or

- recognize that a nothing cannot be in a something:

# flags is O_RDONLY
if O_RDONLY in flags:  # true
pass

# flags is O_RDRW
if O_RDONLY in flags:  # false
pass

# flags is O_RDRW | O_TRUNC
if O_RDONLY in flags:  # false
pass

--
~Ethan~



[1] https://en.wikipedia.org/wiki/Vacuous_truth  -- last line of first paragraph
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I5TNPPPZCUCGKRH34NAZIDL6EPUDVVKQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Hasan Diwan]

On Tue, 9 Nov 2021 at 16:01, Bob Fang  wrote:

> This is a modest proposal to consider having sorted containers (
> http://www.grantjenks.com/docs/sortedcontainers/
> )
> in standard library. I know that usually adding stuff to standard library
> requires some strong arguments, so I will try my best to give my reasons
> here:
>

As of 3.7. dicts are sorted[1], but I'm unsure if the order can be
overridden. Indeed:

>>> Boys = {'Tim': 18,'Charlie':12,'Robert':25}

Girls = {'Tiffany':22}


>>> print(cmp(Girls, Boys))

Traceback (most recent call last):

  File "", line 1, in 

NameError: name 'cmp' is not defined

>>> Girls.__gt__(Boys)

NotImplemented


-- H
-- 
OpenPGP: https://hasan.d8u.us/openpgp.asc

If you wish to request my time, please do so using
*bit.ly/hd1AppointmentRequest
*
.
Si vous voudrais faire connnaisance, allez a *bit.ly/hd1AppointmentRequest
*
.

Sent
from my mobile device
Envoye de mon portable
1. https://softwaremaniacs.org/blog/2020/02/05/dicts-ordered/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RNXTH3JRUZ5WH33AKHVOSZF34FGNMS6S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Having Sorted Containers in stdlib?

[Bob Fang]

Hi All,

This is a modest proposal to consider having sorted containers 
(http://www.grantjenks.com/docs/sortedcontainers/ 
) in standard library. I know 
that usually adding stuff to standard library requires some strong arguments, 
so I will try my best to give my reasons here:

1) Some mainstream language support them out of box: C++ for example have 
set/map which are sorted by the key order, and Java has TreeMap which is 
internally a Red-black tree. I understand languages might target different 
audiences, but I think Python’s standard library is quite extensive compared to 
peers. Consider we even have a sqlite driver in the stdlib, I do not think it 
is outrageous to have sorted containers. 
2) These containers are not really easy to implement correctly, and usually is 
out of the scope of day-to-day projects. Especially considering we have a large 
audience of non-hardcore programmers in Python community. They may have the 
need to use these structures, but they do not necessarily have the 
skill/knowledge to implement it. 
3) Granted, people can just pip install this library, but that is one extra 
step and less fraction is better for user experience.
4) These structures are very useful in competitive programming, I know at least 
in Leetcode this library is installed for Python.
5) The said library is of high implementation quality.

I might be stupid here as I am not the maintainer of this library and I might 
be not even in a position to submit it to Python as part of stdlib, but here 
are some of my personal thoughts and would love to hear your opinion!

Thanks!
Bob 

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YB2JD477TKPB2HTXDW6ZXUBD6NFFFHHJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Guido van Rossum]

On Tue, Nov 9, 2021 at 12:27 PM Terry Reedy  wrote:

> On 11/9/2021 1:52 PM, Sebastian Rittau wrote:
> > Am 09.11.21 um 19:26 schrieb Terry Reedy:
> >> The signature of Sebastian's function with honest parameter names is
> >> foo(x_or_y, required_y=_NotGiven, /).  It is the 2nd argument, not the
> >> first, that is optional, as with range.  If required_y is not given,
> >> than x_or_y must be y, and x is given a default that is not part of
> >> the signature because it is explicitly bound when called. If
> >> required_y *is* given, then x_or_y can be x.
> >
> > Just to clarify: This proposal works differently than how range() works.
> > foo(3) would be illegal as the required second parameter ("y") is
> > missing.
>
> No it is not.  If there is one required positional parameter and one
> supplies one positional argument, then that argument must be bound to
> that parameter name.
>

Terry, maybe that is *your* proposal. But Sebastian's proposal works like
he describes. You can argue that there is a problem with those semantics,
but you cannot argue that that is not what Sebastian proposes. And please
remain civil.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PLS4NSRGS3CANFHM6QGHASBSHZEXPYYV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Terry Reedy]

On 11/9/2021 1:52 PM, Sebastian Rittau wrote:

Am 09.11.21 um 19:26 schrieb Terry Reedy:
The signature of Sebastian's function with honest parameter names is 
foo(x_or_y, required_y=_NotGiven, /).  It is the 2nd argument, not the 
first, that is optional, as with range.  If required_y is not given, 
than x_or_y must be y, and x is given a default that is not part of 
the signature because it is explicitly bound when called. If 
required_y *is* given, then x_or_y can be x. 


Just to clarify: This proposal works differently than how range() works. 
foo(3) would be illegal as the required second parameter ("y") is 
missing.


No it is not.  If there is one required positional parameter and one 
supplies one positional argument, then that argument must be bound to 
that parameter name.


This can be solved by either supplying both "x" and "y", e.g. 
foo(None, 3), or by using a named parameter for "y": foo(y=3).


You are asking that 'x' be required if 'y' is passed by position but not 
if 'y' is passed by name.  This is contradictory.


Note that range does not allow passing by keyword and I specified the 
same for foo.  If you make 'y' keyword only, then there is no problem 
making 'x' optional.



the honest names are foo(x=None, y) in my proposal.


No, because you want 'x' is required or not depending on how 'y' is 
passed.



--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UNZM5KMAYQJINBVX5CLBYYFNDU3WWS6L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Sebastian Rittau]

Am 09.11.21 um 19:26 schrieb Terry Reedy:
The signature of Sebastian's function with honest parameter names is 
foo(x_or_y, required_y=_NotGiven, /).  It is the 2nd argument, not the 
first, that is optional, as with range.  If required_y is not given, 
than x_or_y must be y, and x is given a default that is not part of 
the signature because it is explicitly bound when called. If 
required_y *is* given, then x_or_y can be x. 


Just to clarify: This proposal works differently than how range() works. 
foo(3) would be illegal as the required second parameter ("y") is 
missing. This can be solved by either supplying both "x" and "y", e.g. 
foo(None, 3), or by using a named parameter for "y": foo(y=3). Therefore 
the honest names are foo(x=None, y) in my proposal.


 - Sebastian


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MN24EZEYEWUSC2WM2SKXR6BYK3Z3XH4Z/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Typing :: Typed Classifier

[Simon J.M. Kelley]

Hey Rakshith,

On Tue, Nov 9, 2021, at 6:49 PM, rakshith.bhyravabho...@gmail.com wrote:
> Is it  simply "the package has typing"?  Or does it have a more 
> extensive implication?

As far as I can tell it is simply the former.

Regards,

Simon
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EKUBM2YT62INR2V6EMNNXFBD6UJ3ZQBR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Terry Reedy]

On 11/9/2021 9:23 AM, Steven D'Aprano wrote:

By the way, this discussion is probably better suited to the
Python-Ideas mailing list. But since we're here...

On Tue, Nov 09, 2021 at 11:37:40AM +0100, Sebastian Rittau wrote:


To me, the "natural" solution looks like this:

def foo(x=None, y): ...

[...]

Chris Angelico asked:

What would this mean, though:

foo(2)

Is that legal?


No. This would be equal to foo(x=2) (same as now), meaning the required
argument "y" is missing.


That's an odd interpretation. What you described earlier is very similar
to the calling convention of range, which conceptually looks like this:

 range([start=0,] end, [step=1])

With your example of "foo(x=None, y)" I would expect foo(2) to mean that
x gets the default and y gets the passed in argument 2, similar to the
way that range(2) works.


Implementing with *args is possible, but awkward as one must explicitly 
handle each of the 0, 1, 2, 3, and >3 args cases.  The exceptions for 
the 0 and >3 cases should match those given by the normal processing, 
which can change from version to version.


The actual signature is range_(start_stop, stop=_NotGiven, step=1, /), 
as implemented in


_NotGiven = object()
def range_(start_stop, stop=_NotGiven, step=1, /):
if stop is _NotGiven:
start = 0
stop = start_stop
else:
start = start_stop  # Stand-in for actual code.
... return range(start, stop, step)

>>> list(range_(4))
[0, 1, 2, 3]
>>> list(range_(3,5))
[3, 4]
>>> list(range_(3,9,2))
[3, 5, 7]

The signature of Sebastian's function with honest parameter names is 
foo(x_or_y, required_y=_NotGiven, /).  It is the 2nd argument, not the 
first, that is optional, as with range.  If required_y is not given, 
than x_or_y must be y, and x is given a default that is not part of the 
signature because it is explicitly bound when called.  If required_y 
*is* given, then x_or_y can be x.


The reason Python should not accept defaulted args before required args 
is that it would have to create sentinels, rewrite the signature to what 
it actually is, and write the code to bind the input arguments to the 
intended parameters.  Very messy.  Even if possible, adding code would, 
for instance, mess up tracing and disassembly.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FP5VPBODQVAJSVXUIKBSAR4PT2ZMO24S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NASA Inquiry

[Simon J.M. Kelley]

Hi Charita,

On Tue, Nov 9, 2021, at 2:14 PM, Elmore, Charita R. (GSFC-705.0)[TELOPHASE 
CORP] via Python-Dev wrote:
> REF: RITM1146898
> 
> Hello, my name is Charita Elmore and I am a Supply Chain Risk 
> Management Coordinator at NASA.  As such, I ensure that all NASA 
> acquisitions of Covered Articles comply with Section 208 of the Further 
> Consolidated Appropriations Act, 2020, Public Law 116-94, 
> enacted December 20, 2019.  To do so, the Country of Origin (CoO) 
> information must be obtained from the developer of the product(s). 
>  Specifically, identify the country where the original code was 
> developed: 
>  * Homebrew package: mpdecimal 

You might find more information about mpdecimal and its contributors on the
mpdecimal website: https://www.bytereef.org/mpdecimal/index.html
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GLDOM55WIEOCSRD6KLYXBPKCC2DENOAK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Typing :: Typed Classifier

[rakshith . bhyravabhotla]

Hi,

What are the implications of using a Typing :: typed classifier?

Is it  simply "the package has typing"?  Or does it have a more extensive 
implication?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RXMVX2QQNKSGQF5YB6GKXMWA2ESAFKAB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] NASA Inquiry

[Elmore, Charita R. (GSFC-705.0)[TELOPHASE CORP] via Python-Dev]

REF: RITM1146898

Hello, my name is Charita Elmore and I am a Supply Chain Risk Management 
Coordinator at NASA.  As such, I ensure that all NASA acquisitions of Covered 
Articles comply with Section 208 of the Further Consolidated Appropriations 
Act, 2020, Public Law 116-94, enacted December 20, 2019.  To do so, the Country 
of Origin (CoO) information must be obtained from the developer of the 
product(s).  Specifically, identify the country where the original code was 
developed:

  *   Homebrew package: mpdecimal

[cid:image001.png@01D7D539.7FD581F0]

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JPIOC3HXB6CIT4E6PAKA45OEECFAGMN4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Brett Cannon]

On Tue, Nov 9, 2021 at 6:31 AM Steven D'Aprano  wrote:

> By the way, this discussion is probably better suited to the
> Python-Ideas mailing list. But since we're here...
>
> On Tue, Nov 09, 2021 at 11:37:40AM +0100, Sebastian Rittau wrote:
>
> > >>To me, the "natural" solution looks like this:
> > >>
> > >>def foo(x=None, y): ...
> [...]
>
> Chris Angelico asked:
> > >What would this mean, though:
> > >
> > >foo(2)
> > >
> > >Is that legal?
> >
> > No. This would be equal to foo(x=2) (same as now), meaning the required
> > argument "y" is missing.
>
> That's an odd interpretation. What you described earlier is very similar
> to the calling convention of range, which conceptually looks like this:
>
> range([start=0,] end, [step=1])
>
> With your example of "foo(x=None, y)" I would expect foo(2) to mean that
> x gets the default and y gets the passed in argument 2, similar to the
> way that range(2) works.
>

But we all understand range() because we have been using it for years and
there's a mathematical understanding of what it represents. If you were
taught "range(2, 5) returns the numbers 2 through 5, exclusive" and then
were asked, "what does range(2) do?", I'm not sure what a beginner would
assume, but thinking it goes from 5 to infinity wouldn't be an unreasonable
leap to make.

An API like range() is also not pervasive, so it can be supported without
any special parameter support and not feel icky; `def range(x, y=None, z=1,
/)` covers its unique situation.

But for me, the key issue is simply remembering what one argument means
compares to two. foo(1) versus foo(1, 2) is not self-documenting whether
that `1` in the first call would get bound to the same thing as the `1` in
the second call. I would need to have quite the understanding of the API to
know that without having to look something up in the docs. But the current
semantics don't have this issue, and `1` means the same thing in both
scenarios (unless you're doing something really weird like range(), which
is something I wouldn't want to encourage).
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I52V5QLMNR5OUGRUEPX4S6WDCYUF3X2L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: containment and the empty container

[Guido van Rossum]

On Mon, Nov 8, 2021 at 10:29 PM Ethan Furman  wrote:

>
> The way I see it, the following should hold
>
>  empty_flag = RegexFlag(0)
>  any_case = RegexFlag.IGNORECASE
>  any_case_on_any_line = RegexFlag.IGNORECASE | RegexFlag.MULTILINE
>
>  any_case in empty_flag is False
>  any_case_on_any_line in empty_flag is False
>
>  empty_flag in any_case is False
>  empty_flag in any_case_on_any_line is False
>

The latter two defy all logic. Please don't. Your 'in' operator clearly
means "is a subset of", and the empty set emphatically is a subset of all
sets (this is the most basic mainstream set theory you can think of).

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3KNIZZQTSNDNK3BCGFJVPQD7B2PIXEL2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Ignore my previous post Re: containment and the empty container

[Stephen J. Turnbull]

Aargh.

The whole point of Cameron's post was about implementing "in", and all
of it makes sense now.  Sorry.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VG6IR6DZNAMVM6VEZT3AU4MK355SOCSP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: containment and the empty container

[Stephen J. Turnbull]

Cameron Simpson writes:
 > On 08Nov2021 23:32, MRAB  wrote:

 > >A flag could be a member, but could a set of flags?
 > 
 > Probably one flavour should raise a TypeError then with this comparison.  
 > I wouldn't want "member flag present in SomeFlag.something" and 
 > "set-of-members present in SomeFlag.something" to both be valid,

If the latter is valid, so is the former, since "member flag" is
representable as "singleton containing member flag".  (I think this
representation of sets of flags is the primary use case for IntFlag,
as opposed to Flag or IntEnum.)  If I really needed to distinguish, I
wouldn't use a Enum type that can represent multiple members in one
item, I'd use Python sets of Flags, .

 > I would rather punt the "set-of-members present" off to "<"/"<=" if
 > we want to say either (which of course we should be able to say).

 <  == True since IntFlag is derived from int, but
that's not what we want.  You'd have to define __lt__ for every such
flag type.  For convenience, we would want a new Enum type in the
stdlib.

 > Of course, I think resently we don't distinguish a flag combination
 > name (".nothing", ".red_and_blue"==0b11) from single flags
 > overtly.

I think we really don't want to do that in the stdlib.  Such flags
would often support operations like intersection and set difference
that could easily result in singletons, which then might be checked
against an "allow set".  If you care about the distinction between
singleton-representing-element and other sets of flags, elements of
the flag enum and sets of those elements need to be different types.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/U3MK446WGHLNFET55S523NI2JKVMRRTW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Sebastian Rittau]

Am 09.11.21 um 13:44 schrieb Petr Viktorin:
And for the "encoding" case: IMO, varying the return type based on an 
optional "encoding" argument" is a holdover from the pre-typing era, 
when return types were only specified in the documentation -- just 
like "addch" is a holdover from the days when function signatures were 
only described in the docs. Nowadays, I'd consider it bad API design. 
The @overloads are ugly but they work -- just like the API itself. IMO 
we shouldn't add special cases to encourage more of it.


"encoding" arguments might be the most common case, but it's certainly 
not limited to that. And the fact remains that however much we desire it 
to be different, there are loads of APIs out there, both in the standard 
library and in (well received and not so well received) third-party 
libraries that use these constructs. This is not going to change. Also, 
API design is always in the eye of the beholder. Personally I prefer my 
example foo(x=None, y) to the design of range() of using some 
*args/**kwargs hacks as was suggested.


I don't believe Python's users are best served by artificially limiting 
possible API design, especially if it causes genuine problems (as it in 
the case of @overload).


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2FRLNZDC3EUEARCGKAWDCWC66V24L4YK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Steven D'Aprano]

By the way, this discussion is probably better suited to the 
Python-Ideas mailing list. But since we're here...

On Tue, Nov 09, 2021 at 11:37:40AM +0100, Sebastian Rittau wrote:

> >>To me, the "natural" solution looks like this:
> >>
> >>def foo(x=None, y): ...
[...]

Chris Angelico asked:
> >What would this mean, though:
> >
> >foo(2)
> >
> >Is that legal?
> 
> No. This would be equal to foo(x=2) (same as now), meaning the required 
> argument "y" is missing.

That's an odd interpretation. What you described earlier is very similar 
to the calling convention of range, which conceptually looks like this:

range([start=0,] end, [step=1])

With your example of "foo(x=None, y)" I would expect foo(2) to mean that 
x gets the default and y gets the passed in argument 2, similar to the 
way that range(2) works.


-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/S3USWLRSJYLOYMS52UH2YWMFOLTAT7EV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Victor Stinner]

IMO it was a bad idea to merge 2 ncurses C functions into a single
Python function. In the C API, there are two different functions:

* mvwadd_wch(win, y, x, char): 4 arguments
* wadd_wch(win, char): 2 arguments

The Python curses module could/can have a separated function when (y,
x) arguments are needed. Maybe it's not too late to deprecate the
current complex API when passing y and x, and add a new function which
accept (y, x).

As Petr explained, it's a headache to have such very complicated
function prototype :-(

Think about completion in code editor. What is the expected behavior
when pressing TAB key to complete on the code "window.addch(arg, " ?

Victor

On Tue, Nov 9, 2021 at 1:47 PM Petr Viktorin  wrote:
> A more extreme case is functions with an optional *group*. In curses,
> the first two arguments to addch are optional. In `help()` it's
> documented as `window.addch([y, x,] ch[, attr=...])` and you can call it
> as one of:
>
> window.addch(ch)
> window.addch(ch, attr)
> window.addch(y, x, ch)
> window.addch(y, x, ch, attr)
>
> see: https://docs.python.org/3/library/curses.html#curses.window.addch
>
> Supporting this was a headache for Argument Clinic (PEP 436), and AFAIK
> it still isn't possible to express this as an inspect.Signature (PEP 362).
>
> Allowing non-default arguments after default arguments would mean
> introspection tools (and code that uses them) would need to be changed
> to prepare for the new possibilities. It's not free.

-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/47YCD5UA2JGQEVASBNUTAK7GQ4CLVV7E/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Petr Viktorin]

On 09. 11. 21 10:50, Chris Angelico wrote:

On Tue, Nov 9, 2021 at 8:38 PM Sebastian Rittau  wrote:


Currently, Python doesn't allow non-default arguments after default
arguments:

  >>> def foo(x=None, y): pass
File "", line 1
  def foo(x=None, y): pass
   ^
SyntaxError: non-default argument follows default argument

I believe that at the time this was introduced, no use cases for this
were known and this is is supposed to prevent a source of bugs. I have
two use cases for this, one fringe, but valid, the other more important:

The fringe use case: Suppose you have a function that takes a 2D
coordinate value as separate "x" and "y" arguments. The "x" argument is
optional, the "y" argument isn't. Currently there are two ways to do
this, none of them particularly great:

def foo(y, x):  # reverse x and y arguments, confusing
  ...
def foo(x, y=None):  # Treat the x argument as y if only one argument is
provided
  if y is None:
  x, y = y, x
  ...

To me, the "natural" solution looks like this:

def foo(x=None, y): ...
# Called like this:
foo(1, 2)
foo(y=2)

This could also be useful when evolving APIs. For example there is a
function "bar" that takes two required arguments. In a later version,
the first argument gains a useful default, the second doesn't. There is
no sensible way to evolve the API at the moment.



What would this mean, though:

foo(2)

Is that legal? If it is, it has to be the same as foo(y=2), by your
definition. But that would mean that it's hard to get your head around
the mapping of arguments and parameters.

foo(1, 2) # x=1, y=2
foo(1) # x=None, y=1

There are a very very few functions in Python that have this sort of
odd behaviour (range() being probably the only one most programmers
will ever come across), and it's not something to encourage.


A more extreme case is functions with an optional *group*. In curses, 
the first two arguments to addch are optional. In `help()` it's 
documented as `window.addch([y, x,] ch[, attr=...])` and you can call it 
as one of:


window.addch(ch)
window.addch(ch, attr)
window.addch(y, x, ch)
window.addch(y, x, ch, attr)

see: https://docs.python.org/3/library/curses.html#curses.window.addch

Supporting this was a headache for Argument Clinic (PEP 436), and AFAIK 
it still isn't possible to express this as an inspect.Signature (PEP 362).


Allowing non-default arguments after default arguments would mean 
introspection tools (and code that uses them) would need to be changed 
to prepare for the new possibilities. It's not free.



And for the "encoding" case: IMO, varying the return type based on an 
optional "encoding" argument" is a holdover from the pre-typing era, 
when return types were only specified in the documentation -- just like 
"addch" is a holdover from the days when function signatures were only 
described in the docs. Nowadays, I'd consider it bad API design. The 
@overloads are ugly but they work -- just like the API itself. IMO we 
shouldn't add special cases to encourage more of it.



I would instead recommend making the parameters keyword-only, which
would allow any of them to have defaults or not have defaults. In
terms of useful API design, this is usually more helpful than having
an early parameter omitted.


+1. I'm not sure if it's possible to mark args as keyword-only in the 
type stubs while keeping actual implementation backwards-compatible, but 
if it is, it might be a good option.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JIK7EJLN4WG4EJE4UVYVLUORGQB6XQWR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 467: Minor bytes and bytearray improvements

[Victor Stinner]

On Mon, Nov 8, 2021 at 8:21 PM Ethan Furman  wrote:
> The difference with the built-in ascii is the absence of extra quotes and the 
> `b` indicator when a string is used:
>
> ```
>  >>> u_var = u'abc'
>  >>> bytes.ascii(u_var)
> b'abc'

What about bytes, bytearray and memoryview? What is the expected behavior?

I expect that memoryview is not supported (return something like
b''), and that bytes and bytearray are
copied without adding "b" prefix or quotes.

bytes.ascii(b'abc') == b'abc'
bytes.ascii(bytearray(b'abc')) == b'abc'

I just suggest to elaborate the specification in the PEP.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3HJQPZB6QWM7IDPDU3KJ4FVY4ESJHQOK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Paul Moore]

On Tue, 9 Nov 2021 at 10:39, Sebastian Rittau  wrote:

> This might be better API design (although I don't think Python should be
> opinionated about this outside the standard library), but this still
> leaves the API change example and the very real problem of @overloads
> unsolved.

You can handle this using *args and/or **kwargs, so Python does
support this API design if you want to use it. I disagree, however,
with the statement that Python shouldn't be "opinionated" about this.
It's not a matter of being opinionated, IMO, it's about whether Python
has to have explicit language support for every possible use case that
anyone comes up with. In this case, the situation seems rare enough,
and the API design is sufficiently unusual (which is a polite way of
me saying that I think it's a bad design...), that I think it's
entirely reasonable for Python to not support it outside of the
generic *args/**kwargs machinery.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YO35XE6JHYNGQ4JHX3ZI4YBKXJ5XTG4O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Sebastian Rittau]

Am 09.11.21 um 10:50 schrieb Chris Angelico:

On Tue, Nov 9, 2021 at 8:38 PM Sebastian Rittau  wrote:

Currently, Python doesn't allow non-default arguments after default
arguments:

  >>> def foo(x=None, y): pass
File "", line 1
  def foo(x=None, y): pass
   ^
SyntaxError: non-default argument follows default argument

I believe that at the time this was introduced, no use cases for this
were known and this is is supposed to prevent a source of bugs. I have
two use cases for this, one fringe, but valid, the other more important:

The fringe use case: Suppose you have a function that takes a 2D
coordinate value as separate "x" and "y" arguments. The "x" argument is
optional, the "y" argument isn't. Currently there are two ways to do
this, none of them particularly great:

def foo(y, x):  # reverse x and y arguments, confusing
  ...
def foo(x, y=None):  # Treat the x argument as y if only one argument is
provided
  if y is None:
  x, y = y, x
  ...

To me, the "natural" solution looks like this:

def foo(x=None, y): ...
# Called like this:
foo(1, 2)
foo(y=2)

This could also be useful when evolving APIs. For example there is a
function "bar" that takes two required arguments. In a later version,
the first argument gains a useful default, the second doesn't. There is
no sensible way to evolve the API at the moment.


What would this mean, though:

foo(2)

Is that legal?


No. This would be equal to foo(x=2) (same as now), meaning the required 
argument "y" is missing.



I would instead recommend making the parameters keyword-only, which
would allow any of them to have defaults or not have defaults. In
terms of useful API design, this is usually more helpful than having
an early parameter omitted.


This might be better API design (although I don't think Python should be 
opinionated about this outside the standard library), but this still 
leaves the API change example and the very real problem of @overloads 
unsolved.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SNH7R36XOG2VZHKJJ6ENN5GR4Y5NULP3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: containment and the empty container

[Stephen J. Turnbull]

Tim Peters writes:
 > [Ethan Furman ]

 > > .. on the other hand, it seems that collections of related flags
 > > are often treated as in set theory, where the empty set is a
 > > member of any non-empty set.
 > 
 > Not how any set theory I've ever seen works: a set S contains the
 > empty set if and only if some member of S _is_ the empty set.  Which
 > is also how Python's frozenset.__contains__ works (a plain Python set
 > can never contain the empty set, because only a frozenset can contain
 > a set (empty or not)).

Maybe Ethan's thinking of one of the standard set-theoretic
constructions of natural numbers? Ie,

0 = {}, 1 = {0} = {{}}, 2 = {0, 1} = {{}, {{}}}, etc.

Steve

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XDCKP55FVS6CLVUYJMQF5BF5HA4TZOK4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Chris Angelico]

On Tue, Nov 9, 2021 at 8:38 PM Sebastian Rittau  wrote:
>
> Currently, Python doesn't allow non-default arguments after default
> arguments:
>
>  >>> def foo(x=None, y): pass
>File "", line 1
>  def foo(x=None, y): pass
>   ^
> SyntaxError: non-default argument follows default argument
>
> I believe that at the time this was introduced, no use cases for this
> were known and this is is supposed to prevent a source of bugs. I have
> two use cases for this, one fringe, but valid, the other more important:
>
> The fringe use case: Suppose you have a function that takes a 2D
> coordinate value as separate "x" and "y" arguments. The "x" argument is
> optional, the "y" argument isn't. Currently there are two ways to do
> this, none of them particularly great:
>
> def foo(y, x):  # reverse x and y arguments, confusing
>  ...
> def foo(x, y=None):  # Treat the x argument as y if only one argument is
> provided
>  if y is None:
>  x, y = y, x
>  ...
>
> To me, the "natural" solution looks like this:
>
> def foo(x=None, y): ...
> # Called like this:
> foo(1, 2)
> foo(y=2)
>
> This could also be useful when evolving APIs. For example there is a
> function "bar" that takes two required arguments. In a later version,
> the first argument gains a useful default, the second doesn't. There is
> no sensible way to evolve the API at the moment.
>

What would this mean, though:

foo(2)

Is that legal? If it is, it has to be the same as foo(y=2), by your
definition. But that would mean that it's hard to get your head around
the mapping of arguments and parameters.

foo(1, 2) # x=1, y=2
foo(1) # x=None, y=1

There are a very very few functions in Python that have this sort of
odd behaviour (range() being probably the only one most programmers
will ever come across), and it's not something to encourage.

I would instead recommend making the parameters keyword-only, which
would allow any of them to have defaults or not have defaults. In
terms of useful API design, this is usually more helpful than having
an early parameter omitted.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5DPSPZFJBAIWS4UARIJYRCLNVUIUFKWD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Proposal: Allow non-default after default arguments

[Sebastian Rittau]

Currently, Python doesn't allow non-default arguments after default 
arguments:


>>> def foo(x=None, y): pass
  File "", line 1
    def foo(x=None, y): pass
 ^
SyntaxError: non-default argument follows default argument

I believe that at the time this was introduced, no use cases for this 
were known and this is is supposed to prevent a source of bugs. I have 
two use cases for this, one fringe, but valid, the other more important:


The fringe use case: Suppose you have a function that takes a 2D 
coordinate value as separate "x" and "y" arguments. The "x" argument is 
optional, the "y" argument isn't. Currently there are two ways to do 
this, none of them particularly great:


def foo(y, x):  # reverse x and y arguments, confusing
    ...
def foo(x, y=None):  # Treat the x argument as y if only one argument is 
provided

    if y is None:
    x, y = y, x
    ...

To me, the "natural" solution looks like this:

def foo(x=None, y): ...
# Called like this:
foo(1, 2)
foo(y=2)

This could also be useful when evolving APIs. For example there is a 
function "bar" that takes two required arguments. In a later version, 
the first argument gains a useful default, the second doesn't. There is 
no sensible way to evolve the API at the moment.


The more important use case involves @overloads. A condensed example of 
a function where the return type depends on an "encoding" parameter, 
followed by further parameters that could be called like this:


foo(123, "utf-8")  # would return bytes
foo(encoding="utf-8")
foo(123, None)  # would return str
foo(encoding=None)
foo(x=123)  # would return str
foo()

This could ideally be written as:

@overload
def foo(x: int = ..., encoding: None = ...) -> str: ...
@overload
def foo(x: int = ..., encoding: str) -> bytes: ...
# plus the actual implementation

But due to the syntax constraint, this needs to be hacked around with a 
third overload:


@overload
def foo(x: int = ... encoding: None = ...) -> str: ...
@overload
def foo(x: int, encoding: str) -> bytes: ...  # for foo(123, "utf-8")
@overload
def foo(*, encoding: str) -> bytes: ...  # for foo(encoding="utf-8")

Not only is this hard to read, real examples in typeshed are usually 
more complex, with many arguments before or after the affected argument 
or even multiple affected arguments. This often becomes too complex to 
write or maintain. Here is one example from the wild: 
https://github.com/python/typeshed/blob/b95b729b9e07ab21d252701af0f5b7404672b952/stubs/redis/redis/client.pyi#L51


Allowing non-default arguments after default arguments would solve both 
use cases above and eliminates a special case. I'm also not sure what 
exactly the current SyntaxError really protects us from. Adding a 
non-default after a default argument can't really lead bugs.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7W5RMJVBRN6NE6A3LUP5Y4BMOZKQRYWH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: containment and the empty container

[Greg Ewing]

On 9/11/21 7:29 pm, Ethan Furman wrote:
I've had a couple people tell me that they think of flags as sets, and 
use set theory / set behavior to understand how flags and groups of 
flags should interact.


If you're going to think of flags as sets, then 'i in flags' should
be equivalent to '((1 << i) & flags) != 0'.

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/43SHTKMYRYSRK6P2DPG4XVZMRPOSB42E/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: containment and the empty container

[Paul Moore]

On Tue, 9 Nov 2021 at 06:31, Ethan Furman  wrote:

(IMO, Steven D'Aprano's comment above, which I don't think you have
responded to yet, is the clearest summary of the options here. I agree
with what he says, but just wanted to clarify one point below).

> I've had a couple people tell me that they think of flags as sets, and use 
> set theory / set behavior to understand how
> flags and groups of flags should interact.  My main concern is whether that 
> is the only correct way to look at flags,
> and whether the behavior that seems logical to me with regards to, for 
> example, RegexFlag(0), is wrong, unusual, or
> perfectly natural.

Looking at flags as sets, you need to make a very clear distinction¹
between a *group* of flag values (a set) and a *single* flag value (an
element). The operation "in" on sets tests if an *element* is a member
of a *set*. In the examples you are giving, you seem to be confusing
sets with elements - if you have flags 1 (0b001), 2 (0b010) and 4
(0b100), then asking if 3 is "in" 1|2|4 makes no sense, because 3 is a
*set* of flags, and sets don't go on the LHS of the "in" operator.
Asking if 1 is "in" 3 makes some level of sense (element in set) and
is true, but it looks much less convincing when expressed as numbers
rather than named values (where the names can be chosen to make 1
"look like" a value and 3 "look like" a set).

¹ Pedantry digression - in pure set theory, there are *only* sets, and
sets are members of other sets. But this is largely a technical detail
(and even then, some set theories have the idea of "levels" of sets,
which formalises the whole set vs element idea). Real-world intuitions
about sets tend to have sets and elements as different concepts.

> The way I see it, the following should hold
>
>  empty_flag = RegexFlag(0)
>  any_case = RegexFlag.IGNORECASE
>  any_case_on_any_line = RegexFlag.IGNORECASE | RegexFlag.MULTILINE
>
>  any_case in empty_flag is False
>  any_case_on_any_line in empty_flag is False
>
>  empty_flag in any_case is False
>  empty_flag in any_case_on_any_line is False

You seem to be confusing "in" with "is a subset of" here. Strong -1 on
using "in" to mean "is a subset", as that breaks the pattern used for
every other type in Python. None of the examples you give here make
any sense to me, either in terms of a "natural reading" nor in terms
of "flags are like sets and their elements". You also seem to be
confusing a flag value with the 1-element *set* of values containing
just that value².

The use of the name empty_flag for the value RegexFlag(0) makes things
even more confusing for me, as it's emphatically *not* a single flag,
but a collection (specifically the empty collection) of flags.

If I had to think of the flag type strictly as a set type, I'd think
of all the values as sets of flags (even RegexFlag.IGNORECASE would be
"a set of one flag") and the operation you're talking about here would
then be "subset" - which I'd either spell as "<=" or I wouldn't use an
operator for *at all*. I could just barely deal with thinking of
individual bits as having a dual "set or element" nature, which would
make "individual_flag in flag_set" make sense to me, but it *only*
makes sense if the LHS has exactly one bit set. And I think this is
where the confusion comes in - we typically don't think of operators
as only being meaningful for certain values of one operand (well,
there's 1/0, so I guess we could have "Flag(0) in Flag(1)|Flag(2)"
raise ValueError, by analogy with ZeroDivisionError, but that doesn't
seem helpful in any practical situation).

Overall, my view is that we shouldn't support "in" on enum.Flag types
at all, as the potential for confusion is too high. I'm -0 on having
an explicit "is subset" operator, but if we do, it should be named
"<=", not "in". I fear that any attempt to document an "is subset"
operator would get bogged down in confusion over whether a flag value
is a value or a set of values (or would completely confuse people with
a background in set theory, by making it look like "element is subset
of set" was somehow a thing).

² Pedantry digression 2 - set theories that allow values that are
equal to the set containing just that value are possible (such values
are called "Quine atoms" - see
https://math.stackexchange.com/questions/2389726/can-one-element-set-be-considered-equal-to-its-element)
but such theories are not mainstream, and are not as intuitive as
you'd hope - for example, "RegexFlag.IGNORECASE in
RegexFlag.IGNORECASE" just seems weird to me...

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EHPB5775QZOWUIS3LJPISQ6FN4HBRJUG/
Code of Conduct: http://python.org/psf/codeofconduct/