[Python-ideas] Re: factory for sentinel objects
There is already https://pypi.org/project/sentinel/ https://pypi.org/project/sentinels/ Though, I think this should become part of the standard library. It's a fundamental concept, somewhat analogous to namedtuples, enums and dataclass (only a bit less used, but also less complex). Once figured out, the implementation will be very stable. I believe it's not good to depend for such small and stable functionality on third party packages. At least, I'd often then just stay with the sentinel = object() approach. I think https://peps.python.org/pep-0661/ is going very much in the right direction. Will continue discussing in this context. > Matthias Görgens hat am 31.08.2023 11:29 CEST > geschrieben: > > > Seems nice. Just write a library and upload it to one of the usual places? > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/4L7EU26NWJGUZPZ3W5WXRGQJLKNRSLDH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] factory for sentinel objects
The standard pattern to create a sentinel in Python is
>>> Unset = object()
While this is often good enough, it has some shortcomings:
- repr(Unset) is unhelpful:
- copy/deepcopy create a copy of the sentinel object, which can lead to
surprising results such as:
>>> d = {'val': Unset}
>>> d['val'] is Unset
True
>>> d2 = copy.deepcopy(d)
>>> d2['val'] is Unset
False
- The code "Unset = object()" is quite obscure for people not familiar with the
sentinel pattern.
I propose to provide a sentinel() factory function so that
>>> Unset = sentinel("Unset")
>>> repr(unset)
>>> copy.copy(Unset) is Unset
True
>>> copy.deepcopy(Unset) is Unset
True
A simple implementation would be something like
def sentinel(name):
return type(name, (), {
'__repr__': lambda self: f"<{self.__class__.__name__}>",
'__copy__': lambda self: self,
'__deepcopy__': lambda self, memo: self,
})
Likely, the implementation should be refined a bit more.
While it's not rocket science, getting sentinels correct is cumbersome for end
users. Providing such a function in the standard library is only a minor
maintainance burden, but a significant help for users working with sentinels.
Thanks for your consideration!
Tim___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/BXTIVPVDHW644AX3EAJU36A2GFCPX3QS/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: factory for sentinel objects
Sorry, minor bug in the example implementation:
def sentinel(name):
cls = type(name, (), {
'__repr__': lambda self: f"<{self.__class__.__name__}>",
'__copy__': lambda self: self,
'__deepcopy__': lambda self, memo: self,
})
return cls()
> Tim Hoffmann hat am 31.08.2023 10:44 CEST
> geschrieben:
>
>
> The standard pattern to create a sentinel in Python is
>
> >>> Unset = object()
>
> While this is often good enough, it has some shortcomings:
>
> - repr(Unset) is unhelpful:
>
> - copy/deepcopy create a copy of the sentinel object, which can lead to
> surprising results such as:
> >>> d = {'val': Unset}
> >>> d['val'] is Unset
> True
> >>> d2 = copy.deepcopy(d)
> >>> d2['val'] is Unset
> False
>
> - The code "Unset = object()" is quite obscure for people not familiar with
> the sentinel pattern.
>
>
> I propose to provide a sentinel() factory function so that
>
> >>> Unset = sentinel("Unset")
> >>> repr(unset)
>
> >>> copy.copy(Unset) is Unset
> True
> >>> copy.deepcopy(Unset) is Unset
> True
>
>
> A simple implementation would be something like
>
> def sentinel(name):
> return type(name, (), {
> '__repr__': lambda self: f"<{self.__class__.__name__}>",
> '__copy__': lambda self: self,
> '__deepcopy__': lambda self, memo: self,
> })
>
> Likely, the implementation should be refined a bit more.
>
>
> While it's not rocket science, getting sentinels correct is cumbersome for
> end users. Providing such a function in the standard library is only a minor
> maintainance burden, but a significant help for users working with sentinels.
>
> Thanks for your consideration!
> Tim
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/UJ7JOMBIH64SBP6LHXFPNUCOJSSZHE47/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: synatx sugar for quickly run shell command and return stdout of shell command as string result
And also IPython: https://ipython.readthedocs.io/en/stable/interactive/shell.html e.g. ls_lines = !ls -l ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VKYGULDUZ4SHFILFZPM7V6BILJTO57T7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: We should have an explicit concept of emptiness for collections
Ok, I have no problem ignoring PEP-8 where it's not applicable. I've brought this topic up, because I thought there could be a improvement either in PEP-8 and/or by adding something to the language. I still do think that, but I accept that the problem it solves is not considered relevant enough here to take further action. Anyway, thanks all for the discussion! Tim ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/HXW4IDNA5D4XBVJROGBKM2XQ6GTG5IVJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: We should have an explicit concept of emptiness for collections
Guido van Rossum wrote: > So then the next question is, what's the use case? What code are people > writing that may receive either a stdlib container or a numpy array, and > which needs to do something special if there are no elements? Maybe > computing the average? AFAICT Tim Hoffman (the OP) never said. There's two parts to the answer: 1) There functions e.g. in scipy and matplotlib that accept both numpy arrays and lists of flows. Speaking from matplotlib experience: While eventually we coerce that data to a uniform internal format, there are cases in which we need to keep the original data and only convert on a lower internal level. We often can return early in a function if there is no data, which is where the emptiness check comes in. We have to take extra care to not do the PEP-8 recommended emptiness check using `if not data`. 2) Even for cases that cannot have different types in the same code, it is unsatisfactory that I have to write `if not seq` but `if len(array) == 0` depending on the expected data. IMHO whatever the recommended syntax for emptiness checking is, it should be the same for lists and arrays and dataframes. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/Q6KZEXFLJ6TEFSDQM3SXXIVGNFNURPYT/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: We should have an explicit concept of emptiness for collections
Paul Moore wrote: > > **How do you check if a container is empty?** > > IMHO the answer should not depend on the container. While emptiness may > > mean different things for different types. The check syntax can and should > > still be uniform. > > I will note that if we take things to extremes, that constraint simply > cannot be adhered to - types can define bool, len, and any other > special method we care to invent, however they like. With that in > mind, I'd argue that if a collection defines bool and len in such a > way that "not bool(c)" or "len(c) == 0" doesn't mean "c is empty", > then it is a special case, and has deliberately chosen to be a special > case. I can agree that "len(c) == 0" should mean "c is empty" for all practical purposes (and numpy and pandas conform with that). But "bool(c)" should mean "c is empty" is an arbitrary additional constraint. For some types (e.g. numpy, padas) that cannot be fulfilled in a logically consisent way. By requiring that additional constraint, the Python languages forces these containers to be a special case. To sum things up, I've again written the Premise: It is valuable for the language and ecosystem to not have this special casing. I.e. I don't want to write `if not seq` on the one hand and `if len(array) == 0`. The syntax for checking if a list or an array is empty should be the same. Conclusion: If so, PEP-8 has to be changed because "For sequences, (strings, lists, tuples), use the fact that empty sequences are false:" is not a universal solution. The question then is, what other syntax to use for an emptiness check. Possible solutions: 1) The length check is a possible solution: "For sequences, (strings, lists, tuples), test emptiness by `if len(seq) == 0`. N.b. I think this is more readable than the variant `if not len(seq)` (but that could be discussed as well). 2) The further question is: If we change the PEP 8 recommendation anyway, can we do better than the length check? IMHO a length check is semantically on a lower level than an empty-check. Counting elements is a more detailed operation than only checking if we have any element. That detail is not needed and distracting if we are only interested in is_empty. This is vaguely similar to iterating over indices (`for i in range(len(users))`) vs. iterating over elements (`for user in users`). We don't iterate over indices because that's usually a detail we don't need. So one can argue we shouldn't count elements if we only wan to know if there are any. If we come to the conclusion that an explicit empty check is better than a length=0 check, there are again different ways how that could be implmented again. My favorite solution now would be adding `is_empty()` methods to all standard containers and encourage numpy and pandas to add these methods as well. (Alternatively an empty protocol would be a more formal solution to an explicit check). ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FDAOJU7LYZK32VZ53DF4GMWK35NKLGJJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: We should have an explicit concept of emptiness for collections
Ethan Furman wrote: > On 8/24/21 3:03 PM, Tim Hoffmann via Python-ideas wrote: > > **How do you check if a container is empty?** > > IMHO the answer should not depend on the container. > I think this is the fly in the ointment -- just about everything, from len() > to bool(), to add, to iter() /all/ depend > on the container -- even equality depends on the container. `and`, `or`, and > `not` partially depend on the container > (via bool()). Only `is` is truly independent. Sorry, I think you got me wrong: The meaning and thus implementation depends on the type (e.g. each container defines its own __len__()). However the syntax for querying length that is still uniformly `len(container)`. Likewise I'd like to have a uniform syntax for emptiness, and not different syntax for different types (`if not container` / `if len(array) == 0` / `if dataframe.empty`). ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LRK6HHQKBH2Y4USB7DAJNOUF5NE62ZYD/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: We should have an explicit concept of emptiness for collections
I also have the feeling that this is going round in circles. So let me get back to the core question: **How do you check if a container is empty?** IMHO the answer should not depend on the container. While emptiness may mean different things for different types. The check syntax can and should still be uniform. Not a solution: 0) The current `if not seq` syntax. "check Falsiness instead of emptiness" is a simplification, which is not always possible. Possible solutions: 1) Always use `if len(seq) == 0`. I think, this would works. But would we want to write that in PEP-8 instead of `if not seq`? To me, this feels a bit too low level. 2) A protocol would formalize that concept by building respective syntax into the language. But I concede that it may be overkill. 3) The simple solution would be to add `is_empty()` methods to all stdlib containers and encourage third party libs to adopt that convention as well. That would give a uniform syntax by convention. Reflecting the discussion in this thread, I now favor variant 3). Tim ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RNI2JYDY44LDV7LAGNAFT7IJD5CJT2GX/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: We should have an explicit concept of emptiness for collections
Christopher Barker wrote: > But I see no reason to add a standardized way to check for an empty > container- again “emptiness” may not be obviously defined either. > Numpy arrays, (or Pandas Dataframes) are a good example here — there are > more than one way to think of them as false - but maybe more than one way > to think of them as empty too: > Is a shape () (scalar) array empty ? > Or shape (100, 0) ? > Or shape (0, 100) > Or shape (0,0,0,0) > Or a rank 1 array that’s all zeros? Or all false? > Anyway, the point is that “emptiness” may be situation specific, just like > truthiness is. Just like length is. It's a basic concept and like __bool__ and __len__ it should be upon the objects to specify what empty means. > So explicitly specifying that you are looking for len(container) == 0 is > more clear than isempty(container) would be, even if it did exist. As written in another post here, `len(container) == 0` is on a lower abstraction level than `isempty(container)`. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/BGFQYXXQL6Z2Y4GXCOX7OB55RAKQMVFE/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: We should have an explicit concept of emptiness for collections
Ethan Furman wrote: > On 8/23/21 2:31 PM, Tim Hoffmann via Python-ideas wrote: > > Ethan Furman wrote: > > > It seems to me that the appropriate fix is for numpy to have an > > > "is_empty()" function > > that knows how to deal with arrays and array-like structures, not force > > every container > > to grow a new method. > > Yes, numpy could and probably should have an "is_empty()" method. However, > > defining a > > method downstream breaks duck typing and maybe even more important authors > > have to > > mentally switch between the two empty-check variants `if users` and `if > > users.is_empty()` > > depending on the context. > > The context being whether or not you working with numpy? Is there generic > > code that works with both numpy arrays and > other non-numpy data? Do we have any non-numpy examples of this problem? E.g. SciPy and Matplotlib accept array-like inputs such as lists of numbers. They often convert internally to numpy arrays, but speaking for the case of Matplotlib, we sometimes need to preseve the original data and/or delay conversion, so we have to take extra care when checking inputs. Pandas has the same Problem. "The truth value of a DataFrame is ambiguous". They have introduced a `DataFrame.empty` property. I suspect the same problem exists for other types like xarray or tensorflow tensors, but I did not check. > > If you want to write a function that accepts array-like `values`, you have > > to change > > a check `if values` to `if len(values) == 0`. That works for both but is > > against the > > PEP8 recommendation. This is a shortcoming of the language. > > Numpy is not Python, but a specialist third-party package that has made > > specialist > > choices about basic operations -- that does not sound like a shortcoming of > > the language. > > The "specialist choices" ``if len(values) == 0` in Numpy are the best you > > can do within > > the capabilities of the Python language if you want the code to function > > with lists and > > arrays. For Numpy to do better Python would need to either provide the > > above mentioned > > "has element-wise operations" protocol or an is_empty protocol. > > I consider emptiness-check a basic concept that should be consistent and > > easy to use across containers. > Python has an emptiness-check and numpy chose to repurpose it -- that is not > Python's problem nor a shortcoming in Python. Python does not have an emptiness-check. Empty containers map to False and we suggest in PEP8 to use the False-check as a stand in for the emptiness-check. But logically falsiness and emptiness are two separate things. Numpy (IMHO with good justification) repurposed the False-check, but that left them without a standard emptiness check. > Suppose we add an `.is_empty()` method, and five years down the road another > library repurposes that method, that other > library then becomes popular, and we then have two "emptiness" checks that > are no longer consistent -- do we then add a > third? I don't think this is a valid argument. We would introduce the concept of emptiness. That can happen only once. What empty means for an object is determined by the respective library and implemented there, similar to what `len` means. There can't be any inconsistency. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/CPW43PRUJ7X4Y3SWMTA26LIPKAECAV55/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: We should have an explicit concept of emptiness for collections
Ethan Furman wrote: > It seems to me that the appropriate fix is for numpy to have an "is_empty()" > function that knows how to deal with arrays > and array-like structures, not force every container to grow a new method. Yes, numpy could and probably should have an "is_empty()" method. However, defining a method downstream breaks duck typing and maybe even more important authors have to mentally switch between the two empty-check variants `if users` and `if users.is_empty()` depending on the context. Ethan Furman wrote: > On 8/23/21 1:15 PM, Tim Hoffmann via Python-ideas wrote: > > If you want to write a function that accepts array-like `values`, you have > > to change > > a check `if values` to `if len(values) == 0`. That works for both but is > > against the > > PEP8 recommendation. This is a shortcoming of the language. > > Numpy is not Python, but a specialist third-party package that has made > > specialist choices about basic operations -- > that does not sound like a shortcoming of the language. The "specialist choices" ``if len(values) == 0` in Numpy are the best you can do within the capabilities of the Python language if you want the code to function with lists and arrays. For Numpy to do better Python would need to either provide the above mentioned "has element-wise operations" protocol or an is_empty protocol. I consider emptiness-check a basic concept that should be consistent and easy to use across containers. Tim ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/7FMYUH7L22LQOIV5EZKW56GFRKFQZQND/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: We should have an explicit concept of emptiness for collections
> What's the problem being solved by isempty? Are there any situations > that couldn't be solved by either running a type checker, or by using > len instead of bool? I agree that determining the type is possible most of the time, either by type hints or a static analyzer. Using len is possible, with the limitation that you need a full `len(x) == 0` for numpy arrays (see discussion above). The type aspect was emphasized in the video. I'm not too worried about that explicitly. The video was more of a starting point for me to reconsider the ideom `if not users`. My conclusion (and thus proposal) differs from the video. On a technical level, everything can be solved with the current language capabilities. The main advantage is clearer semantics (explicit is better / readability counts): - Re bool: As experienced python users we are used to translate `if not users` to "if users is empty" or "if we have no users", but it *is* less explicit than `if users.is_empty()`. - Re len: `if not len(users)` or `if len(users) == 0` is more explicit, but its semantically on a lower level. Counting elements is a more detailed operation than only checking if we have any element. That detail is not needed and distracting if we are only interested in is_empty. This is vaguely similar to iterating over indices (`for i in range(len(users))`) vs. iterating over elements (`for user in users`). We don't iterate over indices because that's usually a detail we don't need. I acknowledge that discussing readability can be vague and subjective, not least because we are used to the current ideoms. I'm also aware that we should be very conservative on adding new API, in particular if it's not technically necessary. However, Python makes a strong point on readability, which IMHO is one of the major reasons for its success. I belive that in that context adding is_empty syntactic sugar would be a clear improvement. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/JQTXNHGJZYQL3BWNCED5WAJG2KITCFTG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: We should have an explicit concept of emptiness for collections
Steven D'Aprano wrote: > I don't think it is very common to use numpy arrays in a context where > they are expected to duck-type as collections. Maybe not "numpy arrays duck-type as collections", but it is very common that arrays and sequences are used interchangably. Numpy has created the term "array-like" for this. Well technically, an array-like is something that `np.array()` can turn into an array. But from a user point of view array-like effectively means array or sequence. If you want to write a function that accepts array-like `values`, you have to change a check `if values` to `if len(values) == 0`. That works for both but is against the PEP8 recommendation. This is a shortcoming of the language. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ETQ4UJTVQLW2WNQWZTWKBWGSY2JRHRA4/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] We should have an explicit concept of emptiness for collections
Hi all,
The Programming Recommendations section in PEP-8 states
"For sequences, (strings, lists, tuples), use the fact that empty sequences are
false:"
# Correct:
if not seq:
if seq:
# Wrong:
if len(seq):
if not len(seq):
In the talk "When Python Practices Go Wrong" Brandon Rhodes makes a good point
against this practice based on "explicit is better than implicit"
(https://youtu.be/S0No2zSJmks?t=873). He advertizes using
if len(seq):
While that is as explicit as one can get within the current language, it could
still be more explicit: Semantically, we're not interested in the (zero) length
of the sequence, but want to know if it is empty.
**Proposal**
Therefore, I propose syntax for an explicit empty check
if isempty(seq): (i)
or
if seq.is_empty() (ii)
This proposal is mainly motivated by the Zen verses "Explicit is better than
implicit" and "Readability counts".
The two variants have slightly different advantages and disadvantages:
(i) would add an __isempty__ protocol and a corresponding isempty() builtin
function.
One could argue that this is somewhat redundant with __len__. However, this is
a typical pattern in collection.abc abstract base classes: There are only a few
abstract methods and futher concepts are added as mixin methods (which are by
default implemented using the abstract methods).
https://docs.python.org/3/library/collections.abc.html#collections-abstract-base-classes
(ii) would alternatively only implement a method on the collection. There's
also precedence for predefined methods on collections, e.g. Sequence.index().
Advantages over the protocol approach are:
- It's a smaller language change than adding a protocol and a builtin function
- The order seq.is_empty() matches the order used in spoken english ("if the
sequence is empty") which is more readable than isempty(seq).
- It's tab-completable (-> usability)
Disadvantages:
- Emptiness is similar to length, and people might be used to the
builtin-method concept for such things.
- A protocol is more powerful than a method: One can support isempty() via
len() even for objects that do not implement __isempty__. However, this more a
theoretical advantage. I assume that in practice all collections derive from
the collection.abc classes, so if the is_empty() method is implemented there,
all typical use cases should be covered.
I'm looking forward to your feedback!
Tim
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/PBRSOAFHNT5YWKDPJKSS6SW4JA62CARC/
Code of Conduct: http://python.org/psf/codeofconduct/
