This:
def advance(it, n):
try:
return it[n:]
except TypeError:
return itertools.islice(it, n, None)
has the disadvantages of:
1. Requiring a temporary copy of the data sliced (if len(it) is 1_000_000,
and n is 500_000, you're stuck between 500_000 pointless __next__ calls, a
500_000 long temporary list or inefficiently looking up 500_000 elements by
index)
2. Not working with sequence iterators, only sequences themselves (so if
you want to read 1000, skip 1000, read 1000, skip 1000, over and over, you
can't just use a single stateful iterator for the whole process without
using the consume recipe and calling __next__ 1000 times for each skip)
3. Not working with all sorts of things where a dedicated advance
implementation would allow efficient skipping, e.g. the new insertion
ordered dicts when no deletions have been performed (a state we already
track for other reasons) can advance the iterator (for the raw dict, keys,
values or items) with the same efficiency as sequences do (and could save a
lot of work building and tossing tuples iterating .items() even if it can't
assume no dummy entries); the various itertools functions like
combinations, permutations and combinations_with_replacement (and in some
cases, product) could also be advanced efficiently without the expense of
generating the intermediate states.
Point is, the OP's case (a single sequence, advanced exactly once) is the
only one that implementation addresses, and it still has scaling issues
even then.
On Tue, Oct 6, 2020 at 6:21 PM Chris Angelico <ros...@gmail.com> wrote:
> On Wed, Oct 7, 2020 at 4:53 AM Christopher Barker <python...@gmail.com>
> wrote:
> >
> > On Tue, Oct 6, 2020 at 10:28 AM Alex Hall <alex.moj...@gmail.com> wrote:
> >>
> >>
> >> if you want to iterate through items N to the end, then how do you do
> that without either iterating through the first N and throwing them away,
> or making a slice, which copies the rest of the sequence?
> >>
> >> ```python
> >> for i in range(start, stop):
> >> x = lst[i]
> >> process(x)
> >> ```
> >
> >
> > well yes, of course. but it's kind of a principle in Python that you
> don't use indices to iterate through a sequence :-)
> >
> > And I still like the sequence view idea, because then the creating of
> the "subview" could be done outside the iteration code, which would not
> have to know that it was working with a sequence, rather than any arbitrary
> iterable.
> >
>
> Slices returning views would be elegant, but backward incompatible, so
> it might have to be something like lst.view[start:stop] to create a
> view. I'm not a fan of an "advance this iterator" feature (it'd need a
> dunder on the iterator to make it possible, and I don't see a lot of
> use for it), but perhaps some iterators could themselves support
> slicing. That would be completely backward compatible, since currently
> none of the iterators concerned have any support for indexing. If you
> had that, you could build your own advance() or consume() function,
> something like:
>
> def advance(it, n):
> try:
> return it[n:]
> except TypeError:
> return itertools.islice(it, n, None)
>
> Still unconvinced of its value, but a good use-case would sway me, and
> the C implementation for list_iterator wouldn't be too hard (it'd
> construct a new list_iterator with the same underlying list and an
> incremented index).
>
> ChrisA
> _______________________________________________
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/T52Z25ZMCDVFEWY3W5WMIVO5XSCX46MN/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/RQD2XY5KP5TDU2MJXB5VX2D4IO4EOH5T/
Code of Conduct: http://python.org/psf/codeofconduct/
