[Python-ideas] Re: Enhance list.index with a comparison function
Christopher Barker writes:
> Interesting -- in other recent threads, Ive felt that those of us that
> thought "iterators and `next" were relatively advanced concepts that
> newbies didn't need to learn were dismissed ...
I for one don't *dismiss* that idea iterators and next are advanced,
but I wonder if there aren't halfway explanations for *many* of the
issues that come up that
> doesn't really require the full concept of iterators and iterables
I also believe that *some* of the halfway explanations are useful to
new Pythonistas, and don't (necesssarily) leave them with
misconceptions, such David's implementation of "first_one_like_this"
and Andrew's explanation of files as streams ("they know where they're
going to read next").
(Note: AFAICS the only definition of iterable you need until you've
introduced ABCs and/or protocols is "you can usefully put iterables,
and nothing that isn't iterable, in the 'in' clause of a 'for'
statement". This is much simpler than explaining reasonably
completely what an iterator is and why you might care. It's obvious
why you care about iterables, of course.)
> Interestingly, this seems to contradict API design-wise, what's been pushed
> on recent threads:
>
> * Rather than adding a keyword argument that changes behavior, we should
> add another function to itertools (or the like)
This is a matter of taste. I subscribe to it; I thought Guido did
too, but he says that in this case at least he likes the flag.
> and
>
> * rather than add functionality to a built-in (or ABC), we should add
> utility functions that can act on many related types
This is a principle some subscribe to, yes, but it doesn't apply in
the zip case, since zip already acts on an tuple of arbitrary iterables.
It does apply to ABCs, but it's most persuasive when a highly abstract
ABC already has all the prerequisites for the function to work. For
example, iter() has an obvious implementation for any sequence, it's
basically just
def iterseq(seq):
def wrapper():
for i in range(len(seq)):
yield seq[i]
return wrapper()
and it works because all sequences already have __len__ and
__getitem__. Then for *unordered* containers like sets and dicts, you
can add a protocol (__iter__).
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/PTPCXCJT6EBJ46HQMUD3AWK7GL4ZX5ZH/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Enhance list.index with a comparison function
> > for obj in somelist: > > > if comparison(obj, needle): > > > do_something(obj) > > > break > People who think in functional programming terms will probably love the > `next(filter(...))` idiom, but not everyone thinks or likes functional > programming idioms, and there are many people who would reject a patch > containing that idiom. > I wouldn't reject it, but I don't love the functional idioms (well, I don't love the functional functions (map, filter, vs comprehensions) -- I'd be inclined to go the comprehension way -- which woukld only require understanding next() if you only wanted the first one, but I suspect most of the time you don't only want the first anyway. 1. It requires teaching people about iterators and `next` first. > Interesting -- in other recent threads, Ive felt that those of us that thought "iterators and `next" were relatively advanced concepts that newbies didn't need to learn were dismissed ... But in this case: next(i for i in a_list if pred(i)) doesn't really require the full concept of iterators and iterables -- it requires comprehension syntax, and a quick "next() gives you the next item in an 'iterator' and an 'iterator' is something you can put in a for loop." -- yes, that leaves out all kinds of important details, but gets the job done for now. In fact, in my intro class, I've used "how to get the first item from a thing" as my first introduction to next() :-) 2. If you want to operate on a sublist, you have to teach them about > slicing, so you can introduce itertools.islice, rather than just say > "give the start and end positions as arguments". > hmm -- a great reason to provide an easily accessible "lazy" slice -- like the sequence view proposal recently discussed (and waiting for me to flesh out ...) > 3. If you need the index instead of the value, using filter becomes > downwrite obfuscated: > > next(filter(lambda t: pred(t[1]), enumerate(iterable)))[0] > indeed. but how often DO people need the index? I suspect many uses of .index() are used to then get the object anyway. And I'm not sure it's THAT obfuscated -- we teach folks to use enumerate() pretty early as a way to avoid explicitly looping through indexes. We have a simple answer to a simple question: > > "How do I search a list?" > "Use list.index" > > With this proposal, we get: > > "How do I search a list by some key?" > "Use list.index with a key function" > Interestingly, this seems to contradict API design-wise, what's been pushed on recent threads: * Rather than adding a keyword argument that changes behavior, we should add another function to itertools (or the like) and * rather than add functionality to a built-in (or ABC), we should add utility functions that can act on many related types Granted, this is not a flag (neither boolean, ternary or mode-switching), and it's not a new method, but the API principles may still apply. In particular, a function in itertools would have a lot more utility for various iterables, rather than just lists (or the /sequence ABC?) -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/KU27MEEYBIBWIJOOLTCSIFJWH5RW6YOP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Enhance list.index with a comparison function
On Saturday, May 23, 2020, at 11:02 -0400, David Mertz wrote: > Still, generator comprehension are great. And next() is an excellent > function. Agreed, on both counts. I often end up needing an arbitrary element of a set (or the only element of a single-element set), and next(iter(set)) scratches that itch well. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FQ3H5OEJZOEPVRJ7APTT4EJF2FQZQLBH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Enhance list.index with a comparison function
On Sat, May 23, 2020, 10:54 AM Rob Cliffe via Python-ideas > index_of(needle, haystack, key=func) > > Sounds like a list comprehension: [ needle for needle in haystack if > func(needle) ] > The times it doesn't sound like a list comprehension is when you have a million items in the list, 100k of which match the predicate, but you only care about the first one... Or even just the first ten. Still, generator comprehension are great. And next() is an excellent function. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2ZQR3HMRLIPSOAAK7GJQBJX6MQLUEIKD/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Enhance list.index with a comparison function
On 23/05/2020 05:48, David Mertz wrote: On Sat, May 23, 2020, 12:26 AM Steven D'Aprano Obviously not all such key functions are that simple and you may need to write a helper function, but the same applies to filter. I like the key function much better than the predicate. In large part that's because as soon as you say predicate, I think filter. Particularly if I care about laziness in not looking through extra items. If you wanted everything matching a predicate, using a comprehension just seems more obvious. It could be lazy with a generator comprehension, not them you need next() to get the first. Key= has the obvious parallel with sorted() and friends. Even then, a function similar to sorted() feels more general than a method on lists only. I.e. index_of(needle, haystack, key=func) I'm not sure where that would live exactly, maybe itertools. I think you were on my side in believing an import from standard library is not a huge burden. I think if such a function existed, I'd want another argument to match n != 1 results. But then it gets ugly because one index is an integer, and multiple indices are some sort of collection. Sounds like a list comprehension: [ needle for needle in haystack if func(needle) ] So maybe one is best... ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/W24RTDL4VHKNDMLVMRO6CB3V2AT544XP/ Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/JJIE4L722CUGQF5ACIHT65L6YCJLZYWI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Enhance list.index with a comparison function
On Sat, May 23, 2020, 12:26 AM Steven D'Aprano > Obviously not all such key functions are that simple and you may need to > write a helper function, but the same applies to filter. > I like the key function much better than the predicate. In large part that's because as soon as you say predicate, I think filter. Particularly if I care about laziness in not looking through extra items. If you wanted everything matching a predicate, using a comprehension just seems more obvious. It could be lazy with a generator comprehension, not them you need next() to get the first. Key= has the obvious parallel with sorted() and friends. Even then, a function similar to sorted() feels more general than a method on lists only. I.e. index_of(needle, haystack, key=func) I'm not sure where that would live exactly, maybe itertools. I think you were on my side in believing an import from standard library is not a huge burden. I think if such a function existed, I'd want another argument to match n != 1 results. But then it gets ugly because one index is an integer, and multiple indices are some sort of collection. So maybe one is best... ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/W24RTDL4VHKNDMLVMRO6CB3V2AT544XP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Enhance list.index with a comparison function
On Fri, May 22, 2020 at 06:37:11PM -0700, Ethan Furman wrote:
> On 05/22/2020 05:11 PM, David Mertz wrote:
> >On 05/22/2020 04:43 AM, Steven D'Aprano wrote:
>
> >>i = somelist.index(needle, pred=comparison)
>
> >Why not just this (by object, not by its index, but that seems simpler):
> >
> > >>> do_something(next(filter(pred, somelist)))
> > Something about 55
> > >>> somelist
> > [3, 4, 29, 23, 46, 55, 90, 81]
> > >>> pred
> >
>
> Steven, using David's example list, what would `needle` and `comparison` be
> in your proposal?
Good question! Thank you for pushing me to think about this a little
harder.
I read David's example as "first item that is divisible by 5" so the
predicate would be rubbish:
# I can never remember if the needle comes first or second...
somelist.index(0, pred=lambda a, b: a%5 == b)
somelist.index(0, pred=lambda a, b: b%5 == a)
Yuck. So I think I have the wrong mental model here.
Going back to the thread on discuss that inspired the question, I think
a *key function* is probably better:
for index, blwr in enumerate(blowers):
if blwr.id_ == value:
print(index)
break
# becomes
blowers.index(value, key=operator.attrgetter('id_'))
# or if you prefer
blowers.index(value, key=lambda obj: obj.id_)
although I expect the attrgetter version may be faster, for those who
care about such things.
With a key function, David's example becomes:
somelist.index(0, key=lambda n: n%5)
Here are a few more possibilities.
# number not divisible by 5
somelist.index(True, key=lambda x: bool(x%5))
# case-insensitive search
somelist.index('spam', key=str.casefold)
# number less than 0
somelist.index(True, key=lambda x: x<0)
# item of length exactly 2
somelist.index(2, key=len)
# identity search
somelist.index(True, key=lambda obj: obj is someobj)
# find a non-NAN
somelist.index(False, key=math.isnan)
# find a particular user
somelist.index('guido', key=lambda user: user.name)
# instance of a class
somelist.index(MyClass, key=type) # doesn't match subclasses
# does match subclasses
somelist.index(True, key=lambda obj: isinstance(obj, MyClass))
# z-coordinate of a 3D point equals -7
somelist.index(-7, key=lambda point: point[2])
# sum of values in the item equals 99
somelist.index(99, key=sum)
# sum of sales for the month is greater than 1000
somelist.index(True, key=lambda x: sum(x.sales() > 1000))
Obviously not all such key functions are that simple and you may need
to write a helper function, but the same applies to filter.
Some of these examples are a little artificial, in that you might want
*all* of the items that match, not just the first. In that case, a
filter function is probably better. Or you could loop, adjusting the
start index each time. But if you do want only the first matching value,
or if you know that there is only one such value, then a key function is
more obvious than calling next and filter.
--
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/IVPAOSLKVJTBU367CMA6W2YT3BT7JQ5D/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Enhance list.index with a comparison function
On Fri, May 22, 2020 at 8:59 PM Steven D'Aprano wrote: > > Why not just this (by object, not by its index, but that seems simpler): > > >>> do_something(next(filter(pred, somelist))) > > Sure, that works too. But have you ever written it for real? I haven't. > And having seen it, I'll probably forget all about it within an hour. > Yes, I've written that, or something very similar for real. More than once even. But I do think in functional terms fairly easily. However, even if you don't do it that way, changing the signature on method on `list` specifically feels really clunky. What if I want to do something to the first item in a tuple that meets a condition? Or the first from a deque. Or from a set. Or even the first matching key from a dictionary. Or the first match from some third-party collection? Even if next(filter(...)) isn't how you think, an imperatively written function can be much more generic. For example (untested): def process_first(it, pred=lambda _: True, func=lambda _: _, sentinel=None): for item in it: if pred(item): return func(item) return sentinel Just stick that somewhere, and it does everything you want and more. I use a sentinel rather than raise an exception if nothing matches. That feels much more natural to me, but you can change that if such makes more sense to you. Likewise, if you want to return the "index", it's a simple enough change (but what is the index of an unsized iterable? I mean, of course you can enumerate() or i+=1... but it might not really be an index. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FTKB5JBIJZAWGXIR4MSKVQOTSEPQOZE5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Enhance list.index with a comparison function
On 05/22/2020 05:11 PM, David Mertz wrote: On 05/22/2020 04:43 AM, Steven D'Aprano wrote: i = somelist.index(needle, pred=comparison) Why not just this (by object, not by its index, but that seems simpler): >>> do_something(next(filter(pred, somelist))) Something about 55 >>> somelist [3, 4, 29, 23, 46, 55, 90, 81] >>> pred Steven, using David's example list, what would `needle` and `comparison` be in your proposal? -- ~Ethan~ ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/S2BQMHS3AMCLTCL5XEXXG3NUDD3WROJH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Enhance list.index with a comparison function
On Fri, May 22, 2020 at 08:11:16PM -0400, David Mertz wrote: > > > > After I answered that question, it dawned on me that I have probably > > written something like that loop, or variations of it, a thousand times: > > > > for obj in somelist: > > if comparison(obj, needle): > > do_something(obj) > > break > > > > Why not just this (by object, not by its index, but that seems simpler): > > >>> do_something(next(filter(pred, somelist))) Sure, that works too. But have you ever written it for real? I haven't. And having seen it, I'll probably forget all about it within an hour. People who think in functional programming terms will probably love the `next(filter(...))` idiom, but not everyone thinks or likes functional programming idioms, and there are many people who would reject a patch containing that idiom. Here are some difficulties with the functional version: 1. It requires teaching people about iterators and `next` first. 2. If you want to operate on a sublist, you have to teach them about slicing, so you can introduce itertools.islice, rather than just say "give the start and end positions as arguments". 3. If you need the index instead of the value, using filter becomes downwrite obfuscated: next(filter(lambda t: pred(t[1]), enumerate(iterable)))[0] so it becomes a question of having horses for courses. We have a simple answer to a simple question: "How do I search a list?" "Use list.index" With this proposal, we get: "How do I search a list by some key?" "Use list.index with a key function" -- Steven ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/3J522Z2CG6UQZQVUZEU7DSIYXFHLYC2M/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Enhance list.index with a comparison function
> > After I answered that question, it dawned on me that I have probably > written something like that loop, or variations of it, a thousand times: > > for obj in somelist: > if comparison(obj, needle): > do_something(obj) > break > Why not just this (by object, not by its index, but that seems simpler): >>> do_something(next(filter(pred, somelist))) Something about 55 >>> somelist [3, 4, 29, 23, 46, 55, 90, 81] >>> pred My style works on general iterables, including infinite ones (that hopefully eventually have something fulfilling pred()). -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/P6SZC2PBIEDBHGCAI46YFDD6T4NIA2ZZ/ Code of Conduct: http://python.org/psf/codeofconduct/
