One potentially serious question: what should `enumerate.__reversed__`
> do when given a starting value?
> reversed(enumerate('abc', 1))
> Should that yield...?
> # treat the start value as a start value
> (1, 'c'), (0, 'b'), (-1, 'a')
> # treat the start value as an end value
> (3, 'c'), (2, 'b'), (1, 'a')
> Something else?
> My preference would be to treat the starting value as an ending value.
My idea is largely that as long as any iterable has defined length and
order, it should be reversible, and this should yield the same as if I did
this reverse countdown with some pointer.
It just so happens that `enumerate` depends fully in this regard on the
iterable it enumerates, if it has defined length, then enumerate will also
have defined length. Maybe enumerate should also start defining `len`
whenever underlying iterable supports it.
Regarding the starting value, I think it is important to keep symmetry:
enumerate('abc', 1) is [ (1, 'a'), (2, 'b'), (3, 'c') ], therefore
reversed(enumerate('abc', 1)) should be [(3, 'c'), (2, 'b'), (1, 'a')]
Best Regards,
--
Ilya Kamen
On Thu, Apr 2, 2020 at 5:55 AM <python-dev-requ...@python.org> wrote:
> Send Python-Dev mailing list submissions to
> python-dev@python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> or, via email, send a message with subject or body 'help' to
> python-dev-requ...@python.org
>
> You can reach the person managing the list at
> python-dev-ow...@python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Python-Dev digest..."
>
> Today's Topics:
>
> 1. Re: reversed enumerate (Steven D'Aprano)
> 2. Re: [Python-ideas] Re: reversed enumerate (Andrew Barnert)
> 3. Re: [Python-ideas] Re: reversed enumerate (Andrew Barnert)
>
>
> ----------------------------------------------------------------------
>
> Date: Thu, 2 Apr 2020 13:20:02 +1100
> From: Steven D'Aprano <st...@pearwood.info>
> Subject: [Python-Dev] Re: reversed enumerate
> To: python-dev@python.org
> Cc: python-id...@python.org
> Message-ID: <20200402022000.gc16...@ando.pearwood.info>
> Content-Type: text/plain; charset=us-ascii
>
> Hi Ilya,
>
> I'm not sure that this mailing list (Python-Dev) is the right place for
> this discussion, I think that Python-Ideas (CCed) is the correct place.
>
> For the benefit of Python-Ideas, I have left your entire post below, to
> establish context.
>
> [Ilya]
> > I needed reversed(enumerate(x: list)) in my code, and have discovered
> > that it wound't work. This is disappointing because operation is well
> > defined.
>
> It isn't really well-defined, since enumerate can operate on infinite
> iterators, and you cannot reverse an infinite stream. Consider:
>
> def values():
> while True:
> yield random.random()
>
> a, b = reversed(enumerate(values())
>
> What should the first pair of (a, b) be?
>
> However, having said that, I think that your idea is not unreasonable.
> `enumerate(it)` in the most general case isn't reversable, but if `it`
> is reversable and sized, there's no reason why `enumerate(it)` shouldn't
> be too.
>
> My personal opinion is that this is a fairly obvious and straightforward
> enhancement, one which (hopefully!) shouldn't require much, if any,
> debate. I don't think we need a new class for this, I think enhancing
> enumerate to be reversable if its underlying iterator is reversable
> makes good sense.
>
> But if you can show some concrete use-cases, especially one or two from
> the standard library, that would help your case. Or some other languages
> which offer this functionality as standard.
>
> On the other hand, I think that there is a fairly lightweight work
> around. Define a helper function:
>
> def countdown(n):
> while True:
> yield n
> n -= 1
>
> then call it like this:
>
> # reversed(enumerate(seq))
> zip(countdown(len(seq)-1), reversed(seq)))
>
> So it isn't terribly hard to work around this. But I agree that it would
> be nice if enumerate encapsulated this for the caller.
>
> One potentially serious question: what should `enumerate.__reversed__`
> do when given a starting value?
>
> reversed(enumerate('abc', 1))
>
> Should that yield...?
>
> # treat the start value as a start value
> (1, 'c'), (0, 'b'), (-1, 'a')
>
> # treat the start value as an end value
> (3, 'c'), (2, 'b'), (1, 'a')
>
> Something else?
>
> My preference would be to treat the starting value as an ending value.
>
>
> Steven
>
>
> On Wed, Apr 01, 2020 at 08:45:34PM +0200, Ilya Kamenshchikov wrote:
> > Hi,
> >
> > I needed reversed(enumerate(x: list)) in my code, and have discovered
> that
> > it wound't work. This is disappointing because operation is well defined.
> > It is also well defined for str type, range, and - in principle, but not
> > yet in practice - on dictionary iterators - keys(), values(), items() as
> > dictionaries are ordered now.
> > It would also be well defined on any user type implementing __iter__,
> > __len__, __reversed__ - think numpy arrays, some pandas dataframes,
> tensors.
> >
> > That's plenty of usecases, therefore I guess it would be quite useful to
> > avoid hacky / inefficient solutions like described here:
> > https://code.activestate.com/lists/python-list/706205/.
> >
> > If deemed useful, I would be interested in implementing this, maybe
> > together with __reversed__ on dict keys, values, items.
> >
> > Best Regards,
> > --
> > Ilya Kamen
> >
> > -----------
> > p.s.
> >
> > *Sketch* of what I am proposing:
> >
> > class reversible_enumerate:
> >
> > def __init__(self, iterable):
> > self.iterable = iterable
> > self.ctr = 0
> >
> > def __iter__(self):
> > for e in self.iterable:
> > yield self.ctr, e
> > self.ctr += 1
> >
> > def __reversed__(self):
> > try:
> > ri = reversed(self.iterable)
> > except Exception as e:
> > raise Exception(
> > "enumerate can only be reversed if iterable to
> > enumerate can be reversed and has defined length."
> > ) from e
> >
> > try:
> > l = len(self.iterable)
> > except Exception as e:
> > raise Exception(
> > "enumerate can only be reversed if iterable to
> > enumerate can be reversed and has defined length."
> > ) from e
> >
> > indexes = range(l-1, -1, -1)
> > for i, e in zip(indexes, ri):
> > yield i, e
> >
> > for i, c in reversed(reversible_enumerate("Hello World")):
> > print(i, c)
> >
> > for i, c in reversed(reversible_enumerate([11, 22, 33])):
> >
> > print(i, c)
>
> > _______________________________________________
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/NDDKDUDHE3J7SR7IPO3QXCVFSGE4BAV4/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
> ------------------------------
>
> Date: Wed, 1 Apr 2020 20:42:12 -0700
> From: Andrew Barnert <abarn...@yahoo.com>
> Subject: [Python-Dev] Re: [Python-ideas] Re: reversed enumerate
> To: Steven D'Aprano <st...@pearwood.info>
> Cc: python-dev@python.org, python-id...@python.org
> Message-ID: <647be271-d7ed-4e50-a807-3d748b00f...@yahoo.com>
> Content-Type: text/plain; charset=utf-8
>
> Before jumping in:
>
> In many cases, when you want to reverse an enumerate, it’s small and
> fixed-sized, so there’s a trivial way to do this: Just store the enumerate
> iterator in a tuple, and tuples are reversible.
>
> for idx, value in reversed(tuple(enumerate(stuff))):
>
> But of course there are some cases where this isn’t appropriate, like
> enumerating a fixed-size but huge input.
>
> > On Apr 1, 2020, at 19:23, Steven D'Aprano <st...@pearwood.info> wrote:
> >
> > [Ilya]
> >> I needed reversed(enumerate(x: list)) in my code, and have discovered
> >> that it wound't work. This is disappointing because operation is well
> >> defined.
> >
> > It isn't really well-defined, since enumerate can operate on infinite
> > iterators, and you cannot reverse an infinite stream.
>
> ...
>
> > However, having said that, I think that your idea is not unreasonable.
> > `enumerate(it)` in the most general case isn't reversable, but if `it`
> > is reversable and sized, there's no reason why `enumerate(it)` shouldn't
> > be too.
>
> Agreed—but this is just a small piece of a much wider issue. Today,
> enumerate is always an Iterator. It’s never reversible. But it’s also not
> sized, or subscriptable, or in-testable, even if you give it inputs that
> are. And it’s not just enumerate—the same is true for map, filter, zip,
> itertools.islice, itertools.dropwhile, etc.
>
> There’s no reason these things couldn’t all be views, just like the
> existing dict views (and other things like memoryview and third-party
> things like numpy array slices). In fact, they already are in Swift, and
> will be in C++20.
>
> > My personal opinion is that this is a fairly obvious and straightforward
> > enhancement, one which (hopefully!) shouldn't require much, if any,
> > debate. I don't think we need a new class for this, I think enhancing
> > enumerate to be reversable if its underlying iterator is reversable
> > makes good sense.
>
> Actually, that doesn’t work—it has to be Sized as well.
>
> More generally, it’s rarely _quite_ as simple as just “views support the
> same operations as the things they view”. An enumerate can be a Sequence if
> its input is, but a filter can’t. A map with multiple inputs isn’t
> Reversible unless they’re all not just Reversible but Sized, although a map
> with only one input doesn’t need it to be Sized. And so on. But none of
> these things are hard, it’s just a bunch of work to go through all the
> input types for all the view types and write up the rules. (Or steal them
> from another language or library that already did that work…)
>
> > But if you can show some concrete use-cases, especially one or two from
> > the standard library, that would help your case. Or some other languages
> > which offer this functionality as standard.
>
> Agreed. I don’t think we need to wait until someone designs and writes a
> complete viewtools library and submits it for stdlib inclusion before we
> can consider adding just one extension to one iterator. But I do think we
> want to add the one(s) that are most useful if any, not just whichever ones
> people think of first. I’ve personally wanted to reverse a map or a filter
> more often than an enumerate, but examples would easily convince me that
> that’s just me, and reversing enumerate is more needed.
>
> > One potentially serious question: what should `enumerate.__reversed__`
> > do when given a starting value?
> >
> > reversed(enumerate('abc', 1))
>
> I don’t think this is a problem. When you reversed(tuple(enumerate('abc',
> 1))) today, what do you get? You presumably don’t even need to look that up
> or try it out. It would be pretty confusing if it were different without
> the tuple.
>
>
> ------------------------------
>
> Date: Wed, 1 Apr 2020 20:42:12 -0700
> From: Andrew Barnert <abarn...@yahoo.com>
> Subject: [Python-Dev] Re: [Python-ideas] Re: reversed enumerate
> To: Steven D'Aprano <st...@pearwood.info>
> Cc: python-dev@python.org, python-id...@python.org
> Message-ID: <647be271-d7ed-4e50-a807-3d748b00f...@yahoo.com>
> Content-Type: text/plain; charset=utf-8
>
> Before jumping in:
>
> In many cases, when you want to reverse an enumerate, it’s small and
> fixed-sized, so there’s a trivial way to do this: Just store the enumerate
> iterator in a tuple, and tuples are reversible.
>
> for idx, value in reversed(tuple(enumerate(stuff))):
>
> But of course there are some cases where this isn’t appropriate, like
> enumerating a fixed-size but huge input.
>
> > On Apr 1, 2020, at 19:23, Steven D'Aprano <st...@pearwood.info> wrote:
> >
> > [Ilya]
> >> I needed reversed(enumerate(x: list)) in my code, and have discovered
> >> that it wound't work. This is disappointing because operation is well
> >> defined.
> >
> > It isn't really well-defined, since enumerate can operate on infinite
> > iterators, and you cannot reverse an infinite stream.
>
> ...
>
> > However, having said that, I think that your idea is not unreasonable.
> > `enumerate(it)` in the most general case isn't reversable, but if `it`
> > is reversable and sized, there's no reason why `enumerate(it)` shouldn't
> > be too.
>
> Agreed—but this is just a small piece of a much wider issue. Today,
> enumerate is always an Iterator. It’s never reversible. But it’s also not
> sized, or subscriptable, or in-testable, even if you give it inputs that
> are. And it’s not just enumerate—the same is true for map, filter, zip,
> itertools.islice, itertools.dropwhile, etc.
>
> There’s no reason these things couldn’t all be views, just like the
> existing dict views (and other things like memoryview and third-party
> things like numpy array slices). In fact, they already are in Swift, and
> will be in C++20.
>
> > My personal opinion is that this is a fairly obvious and straightforward
> > enhancement, one which (hopefully!) shouldn't require much, if any,
> > debate. I don't think we need a new class for this, I think enhancing
> > enumerate to be reversable if its underlying iterator is reversable
> > makes good sense.
>
> Actually, that doesn’t work—it has to be Sized as well.
>
> More generally, it’s rarely _quite_ as simple as just “views support the
> same operations as the things they view”. An enumerate can be a Sequence if
> its input is, but a filter can’t. A map with multiple inputs isn’t
> Reversible unless they’re all not just Reversible but Sized, although a map
> with only one input doesn’t need it to be Sized. And so on. But none of
> these things are hard, it’s just a bunch of work to go through all the
> input types for all the view types and write up the rules. (Or steal them
> from another language or library that already did that work…)
>
> > But if you can show some concrete use-cases, especially one or two from
> > the standard library, that would help your case. Or some other languages
> > which offer this functionality as standard.
>
> Agreed. I don’t think we need to wait until someone designs and writes a
> complete viewtools library and submits it for stdlib inclusion before we
> can consider adding just one extension to one iterator. But I do think we
> want to add the one(s) that are most useful if any, not just whichever ones
> people think of first. I’ve personally wanted to reverse a map or a filter
> more often than an enumerate, but examples would easily convince me that
> that’s just me, and reversing enumerate is more needed.
>
> > One potentially serious question: what should `enumerate.__reversed__`
> > do when given a starting value?
> >
> > reversed(enumerate('abc', 1))
>
> I don’t think this is a problem. When you reversed(tuple(enumerate('abc',
> 1))) today, what do you get? You presumably don’t even need to look that up
> or try it out. It would be pretty confusing if it were different without
> the tuple.
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
>
>
> ------------------------------
>
> End of Python-Dev Digest, Vol 201, Issue 1
> ******************************************
>
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/NJLCSLIEITAM2EZOHFFMC2Y6APDPOXY5/
Code of Conduct: http://python.org/psf/codeofconduct/
