[issue44571] itertools: takedowhile()

[paul rubin]

paul rubin  added the comment:

Bah, the above doesn't work in the cases where the iterator is empty or 
(different symptom) where the tail part is empty.  Rather than posting a 
corrected version (unless someone wants it) I'll just say not to use that code 
fragment, but that the intended API still looks reasonable.  I do support 
having some kind of solution but don't want to keep stretching out an already 
closed discussion unless people think there is more to say.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[paul rubin]

paul rubin  added the comment:

Oh wow, before_and_after will go into the itertools module per that patch?  I 
found this issue while looking for a way to this, but had written the following 
implementation:

def span(pred, xs):
# split xs into two iterators a,b where a() is the prefix of xs 
# that satisfies the predicate, and b() is the rest of xs.  
# Similar to Data.List.Span in Haskell. 

ixs = iter(xs)
t = None
def a():
nonlocal t
for x in ixs:
if pred(x): yield x
else: break
t = x
def b():
return itertools.chain([t], ixs)
return a, b

def tspan():  # test
xs = [1,3,5,2,4,6,8]
def odd(x): return x%2==1
# This should print [1,3,5] then [2,4,6,8]  
for p in span(odd, xs):
print(list(p()))

--
nosy: +phr

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[miss-islington]

miss-islington  added the comment:


New changeset 656b0bdfaae3a36d386afe3f7b991744528c3ff7 by Miss Islington (bot) 
in branch '3.10':
bpo-44571:  Add itertool recipe for a variant of takewhile() (GH-28167)
https://github.com/python/cpython/commit/656b0bdfaae3a36d386afe3f7b991744528c3ff7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[Raymond Hettinger]

Change by Raymond Hettinger :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +26600
pull_request: https://github.com/python/cpython/pull/28173

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[Raymond Hettinger]

Raymond Hettinger  added the comment:


New changeset 91be41ad933e24bff26353a19f56447e17fb6367 by Raymond Hettinger in 
branch 'main':
bpo-44571:  Add itertool recipe for a variant of takewhile() (GH-28167)
https://github.com/python/cpython/commit/91be41ad933e24bff26353a19f56447e17fb6367


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[Raymond Hettinger]

Change by Raymond Hettinger :


--
keywords: +patch
pull_requests: +26596
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28167

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[pavel-lexyr]

pavel-lexyr  added the comment:

Thank you - that answers the questions. The use case where we would want to 
know if the last element is transitional or not completely slipped my mind for 
some reason.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[Tim Peters]

Tim Peters  added the comment:

That said, if you really do want those semantics, it's easy to build on top of 
Raymond's API:

def takewhile_plus_one_more_if_any(pred, iterable):
from itertools import islice, chain
before, after = before_and_after(pred, iterable)
return chain(before, islice(after, 1))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[Tim Peters]

Tim Peters  added the comment:

If you don't use the 'after` iterator, then of course you'll never see the 
values (if any) it would have yielded.

How could it possibly be otherwise? By design and construction, the `before` 
iterator ends before yielding the first (if any) transitional element.

As Raymond said at the start, the `takedowhile()` proposal appears much harder 
to use correctly, since there's no reasonably sane way to know that the last 
value it yields _is_ the transitional element (or, perhaps, that there was no 
transitional element, and the underlying iterable was just exhausted without 
finding one).

If the proposal were instead for `takewhile_plus_one_more_if_any()`, then at 
least the ugly name would warn about the surprising intended behavior ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[pavel-lexyr]

pavel-lexyr  added the comment:

There is a core part of the `takedowhile` proposal's use case that I am having 
trouble envisioning via the alternative `before_and_after` proposal. If the 
`after` part of the iterator the user does not engage with, the transitional 
elements will be stuck indefinitely. What would a correct usage be, in case one 
wants the following two conditions to hold true:

1. the amount of elements after the first falsifying one is minimal, i.e. 0
2. all the yielded elements are processed no matter what?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[Tim Peters]

Tim Peters  added the comment:

I agree Raymond's `before_and_after()` looks like an elegant, efficient, and 
usable approach to this.

One minor nit: there's no need for the `iter()` call in:

yield from iter(transition)

Indeed, it confused me at first, because `yield from x` does its own `iter(x)` 
call under the covers, and since everyone knows that ;-) , I wondered what deep 
trickery calling it again was intended to pull off.

But I persuaded myself there was no such subtle intent - it's just redundant.

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> For convenience, the takewhile iterator can also have
> additional attributes: a boolean attribute which indicates 
> that the falsifying element is set, and dynamic attribute 
> which is equal to orig_iterator 
> or chain([odd_element], orig_iterator).

Rather than graft a funky and atypical API onto takewhile(), it would be better 
to have a new tool that returns two iterators, the true steam, and a stream for 
the remaining values.  Either stream may be empty.  There is no need for a 
boolean flag attribute or a remaining stream attribute.  This design fits in 
better with the other itertools.

FWIW, we can already do this using groupby(), but it is only easy if we assume 
the first part of the stream is all true and the remainder of the stream is all 
false.  That isn't good enough for general application.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

For convenience, the takewhile iterator can also have additional attributes: a 
boolean attribute which indicates that the falsifying element is set, and 
dynamic attribute which is equal to orig_iterator or chain([odd_element], 
orig_iterator).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I've done some API experiments using a data munging example.  See attached file.

The proposed API for takewhile() to save the last attribute is somewhat awkward 
to use:

it = iter(report)
tw_it = takewhile(is_header, it)
for line in takewhile(is_header, tw_it):
print('Header:', repr(line))
if hasattr(tw_it, 'odd_element'):
it = chain([tw_it.odd_element], it)
print(mean(map(int, it)))   

What is needed is a new itertool recipe to cover this use case:

headers, data = before_and_after(is_header, report)
for line in headers:
print('Header:', repr(line))
print(mean(map(int, data)))

--
Added file: https://bugs.python.org/file50142/tmp_takewhile.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> What if set the last item as an attribute of the takewhile iterator?

Perhaps raise an attribute error unless the falsifying element is set?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

What if set the last item as an attribute of the takewhile iterator?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[pavel-lexyr]

pavel-lexyr  added the comment:

I see. If the syntax allows for better ways to do it now, perhaps a move 
towards deprecation would be a better idea then? This would agree with the Zen.

Also, please elaborate more on the generator-based solutions you have in mind. 
The suggestion stems from a very real use case - and the lambda function we 
ended up using looks like a poor hack.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[Raymond Hettinger]

Change by Raymond Hettinger :


--
components: +Library (Lib)
versions: +Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Thanks for the suggestion.  I agree that the loss of the non-matching element 
is an irritant.  The suggestion to return the first false element would solve 
that problem but is itself hard to work with.  The result would be difficult to 
reason about because all the elements are except one are true, the last is 
false, and you can't know that you have gotten a false element until one more 
call to next() to determine that no more elements are forthcoming.

Also, I'm reluctant to create any variants for takewhile() or dropwhile().  
Those have been the least successful itertools.  If I had it to do over again, 
they would not have been included.  For the most part, generator based 
solutions are superior in terms of readability, flexibility, and performance.

--
assignee:  -> rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44571] itertools: takedowhile()

[pavel-lexyr]

New submission from pavel-lexyr :

As described in the documentation, itertools.takewhile() returns all the 
elements until the first one that does not match the provided criterion. In 
case of a destructive iterator, or one with side effects, not yielding an 
element downstream may render takewhile() unsuitable for use.

Proposed is itertools.takedowhile() - an alternate function that yields the 
first false element as well, and returns after. The behaviour is identical 
otherwise.

--
messages: 397025
nosy: pavel-lexyr, rhettinger
priority: normal
severity: normal
status: open
title: itertools: takedowhile()
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com