[issue30346] Odd behavior when unpacking `itertools.groupby`

Serhiy Storchaka added the comment: Added the patch for PR 1568 against the current master just for history. -- Added file: https://bugs.python.org/file47167/groupby-invalid.diff ___ Python tracker

[issue30346] Odd behavior when unpacking `itertools.groupby`

Serhiy Storchaka added the comment: Thank you for your review and explanations Raymond. This makes sense to me. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue30346] Odd behavior when unpacking `itertools.groupby`

Serhiy Storchaka added the comment: New changeset c247caf33f6e6000d828db4762d1cb12edf3cd57 by Serhiy Storchaka in branch 'master': bpo-30346: An iterator produced by the itertools.groupby() iterator (#1569) https://github.com/python/cpython/commit/c247caf33f6e6000d828db4762d1cb12edf3cd57

[issue30346] Odd behavior when unpacking `itertools.groupby`

Raymond Hettinger added the comment: Go ahead with PR 1569 to exhaust the inner iterator when the outer iterator advances. Rationale: The OP expected the inner iterator to be exhausted. That is also what the PyPy team originally implemented and it is what I would have expected. Also, there

[issue30346] Odd behavior when unpacking `itertools.groupby`

Serhiy Storchaka added the comment: Ping. Can you make your decision Raymond? This issue is a stopper for issue30347 which fixes a crash. I don't want to merge the fix for issue30347 until this issue be solved, because both proposed PRs solve issue30347 too, and I don't want to make unneeded

[issue30346] Odd behavior when unpacking `itertools.groupby`

Raymond Hettinger added the comment: I've looked at the patches but am still thinking about what I prefer before committing to a re-design. Please give it a little time. There is zero urgency here (nothing is broken, no user complaints, etc). I have a bunch of other stuff on my plate for

[issue30346] Odd behavior when unpacking `itertools.groupby`

Serhiy Storchaka added the comment: Raymond, could you please look at these patches? Both are small. What do you prefer? -- ___ Python tracker ___

[issue30346] Odd behavior when unpacking `itertools.groupby`

Tim Peters added the comment: Users certainly have been fooled by this, although "unpacking" is a red herring. I've been burned by it, and I've seen StackOverflow puzzles related to the same thing. I think this is the heart of it: given a finite iterable I, it usually makes no semantic

[issue30346] Odd behavior when unpacking `itertools.groupby`

Serhiy Storchaka added the comment: PR 1568 and PR 1569 are two subvariants of #1. PR 1568 makes an inner iterator invalid and raising RuntimeError when used. PR 1569 makes an inner iterator exhausted. -- stage: -> patch review ___ Python tracker

[issue30346] Odd behavior when unpacking `itertools.groupby`

Changes by Serhiy Storchaka : -- pull_requests: +1664 ___ Python tracker ___ ___

[issue30346] Odd behavior when unpacking `itertools.groupby`

Changes by Serhiy Storchaka : -- pull_requests: +1663 ___ Python tracker ___ ___

[issue30346] Odd behavior when unpacking `itertools.groupby`

Raymond Hettinger added the comment: > If I was to submit this as a PR, ... Please don't jump to writing code yet. We need to decide what should be done first. I'm not really excited about making a change at all. No users seem to have ever been affected by this. The CPython code matches

[issue30346] Odd behavior when unpacking `itertools.groupby`

Matthew Gilson added the comment: Tracking which group the grouper _should_ be on using an incrementing integer seems to work pretty well. In additional to the tests in `test_itertools.py`, I've gotten the following tests to pass: class TestGroupBy(unittest.TestCase): def

[issue30346] Odd behavior when unpacking `itertools.groupby`

Serhiy Storchaka added the comment: I agree and prefer #1. This will save users from incorrect using and us from reports about odd behavior. The implementation is simple. The groupby object generates an unique identifier when creates a new grouper. This may be a sequential integer, just

[issue30346] Odd behavior when unpacking `itertools.groupby`

Raymond Hettinger added the comment: I have no interest in #2. Itertools are all about *not* storing data in memory unless that is an intrinsic requirement for the operation (tee() and cycle() for example). Also, I would like any changes to be minimal and have the lowest possible risk of

[issue30346] Odd behavior when unpacking `itertools.groupby`

Serhiy Storchaka added the comment: Here is a Python implementation of the idea #2: from itertools import tee def groupby(iterable, key=None): if key is None: key = lambda x: x def grouper(currvalue, it, tgtkey): yield currvalue for currvalue in it:

[issue30346] Odd behavior when unpacking `itertools.groupby`

Serhiy Storchaka added the comment: Does it make sense using a grouper after advancing the groupby iterator? Currently it is possible by accident: >>> from itertools import groupby, count >>> it = groupby(count(), lambda i: (i//10)%2) >>> _, even = next(it) >>> _, odd = next(it) >>>

[issue30346] Odd behavior when unpacking `itertools.groupby`

Matthew Gilson added the comment: Oops. You don't need to pass `self.currvalue` to `_grouper`. I didn't end up using it in there... -- ___ Python tracker

[issue30346] Odd behavior when unpacking `itertools.groupby`

Matthew Gilson added the comment: I think that works to solve the problem that I pointed out. In my stack overflow question (http://stackoverflow.com/a/43926058/748858) it has been pointed out that there are other opportunities for weirdness here. Specifically, if if I skip processing 2

[issue30346] Odd behavior when unpacking `itertools.groupby`

Raymond Hettinger added the comment: I suppose that when the input iterator is exhausted, we could poison the currkey to make the subordinate iterator ends as well: def __next__(self): while self.currkey == self.tgtkey: try: self.currvalue =

[issue30346] Odd behavior when unpacking `itertools.groupby`

Changes by Raymond Hettinger : -- priority: normal -> low ___ Python tracker ___

[issue30346] Odd behavior when unpacking `itertools.groupby`

Raymond Hettinger added the comment: FYI, the CPython behavior matches the pure python implementation show in the docs: Python 3.6.1 (v3.6.1:69c0db5050, Mar 21 2017, 01:21:04) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type "copyright", "credits" or "license()" for more

[issue30346] Odd behavior when unpacking `itertools.groupby`

Changes by Serhiy Storchaka : -- assignee: -> rhettinger nosy: +rhettinger ___ Python tracker ___

[issue30346] Odd behavior when unpacking `itertools.groupby`

New submission from Matt Gilson: There is some odd behavior when unpacking the groups from an itertools.groupby result. For example: from itertools import groupby from operator import itemgetter inputs = ((x > 5, x) for x in range(10)) (_, a), (_, b) = groupby(inputs,