On Tuesday, October 25, 2016 at 6:26:17 PM UTC-4, Nathaniel Smith wrote:
>
> On Sat, Oct 22, 2016 at 9:02 AM, Nick Coghlan > wrote:
> > On 20 October 2016 at 07:02, Nathaniel Smith > wrote:
> >> The first change is to replace the outer for loop with a while/pop
> >> loop, so that if an excep
On 26 October 2016 at 08:48, Nathaniel Smith wrote:
> If it takes a strong reference, then suddenly we're pinning all
> iterators in memory until the end of the enclosing function, which
> will often look like a memory leak. I think this would break a *lot*
> more existing code than the for-scoped
On 26 October 2016 at 08:25, Nathaniel Smith wrote:
> On Sat, Oct 22, 2016 at 9:02 AM, Nick Coghlan wrote:
>> At this point your code is starting to look a whole lot like the code
>> in contextlib.ExitStack.__exit__ :)
>
> One of the versions I tried but didn't include in my email used
> ExitStac
On 26 October 2016 at 01:59, Yury Selivanov wrote:
> But how would it help with a partial iteration over generators
> with a "with" statement inside?
>
>def it():
>with open(file) as f:
>for line in f:
>yield line
>
> Nathaniel proposal addresses this by fix
...Doh. I spent all that time evaluating the function-scoped-cleanup
proposal from the high-level design perspective, and then immediately
after hitting send, I suddenly realized that I'd missed a much more
straightforward technical problem.
One thing that 'with' blocks / for-scoped-iterclose do i
On Sat, Oct 22, 2016 at 9:02 AM, Nick Coghlan wrote:
> On 20 October 2016 at 07:02, Nathaniel Smith wrote:
>> The first change is to replace the outer for loop with a while/pop
>> loop, so that if an exception occurs we'll know which iterables remain
>> to be processed:
>>
>> def chain(*iterables
On 2016-10-25 4:33 AM, Nick Coghlan wrote:
I'm starting to think that we instead need a way
to let them easily say "This resource, the one I just created or have
otherwise gained access to? Link its management to the lifecycle of
the currently running function or frame, so it gets cleaned up wh
On 25 October 2016 at 11:59, Stephen J. Turnbull
wrote:
> On the semantic side,
> constructs like closures and generators (which they may be cargo-
> culting!) mean that it's harder to link resource management to
> (syntactic) function calls than a new developer might think. (Isn't
> that Nathani
On 25 October 2016 at 03:32, Chris Barker wrote:
> On Sat, Oct 22, 2016 at 8:22 PM, Nick Coghlan wrote:
>> The big refactoring benefit that this feature would offer over with
>> statements is that it doesn't require a structural change to the code
>> - it's just wrapping an existing expression in
On 25 October 2016 at 03:16, Chris Barker wrote:
> On Sat, Oct 22, 2016 at 9:17 AM, Nick Coghlan wrote:
>
>>
>> This is actually a case where style guidelines would ideally differ
>> between between scripting use cases ... and
>> library(/framework/application) development use cases
>
>
> Hmm --
Chris Barker wrote:
> Nick Coghlan wrote:
>> Chris, would you be open to trying a thought experiment with some of
>> your students looking at ways to introduce function-scoped
>> deterministic resource management *before* introducing with
>> statements?
I'm with Chris, I think: this seems in
On Sat, Oct 22, 2016 at 8:22 PM, Nick Coghlan wrote:
> Pondering this overnight, I realised there's a case where folks using
> Python primarily as a scripting language can still run into many of
> the resource management problems that arise in larger applications:
> IPython notebooks
> This i
On Sat, Oct 22, 2016 at 9:17 AM, Nick Coghlan wrote:
> This is actually a case where style guidelines would ideally differ
> between between scripting use cases ... and
> library(/framework/application) development use cases
>
Hmm -- interesting idea -- and I recall Guido bringing something lik
On 23 October 2016 at 02:17, Nick Coghlan wrote:
> On 22 October 2016 at 06:59, Chris Barker wrote:
>> And then context managers were introduced. And it seems to be there is a
>> consensus in the Python community that we all should be using them when
>> working on files, and I myself have finally
On 22 October 2016 at 06:59, Chris Barker wrote:
> And then context managers were introduced. And it seems to be there is a
> consensus in the Python community that we all should be using them when
> working on files, and I myself have finally started routinely using them,
> and teaching newbies t
On 20 October 2016 at 07:02, Nathaniel Smith wrote:
> The first change is to replace the outer for loop with a while/pop
> loop, so that if an exception occurs we'll know which iterables remain
> to be processed:
>
> def chain(*iterables):
> try:
> while iterables:
> for el
On Fri, Oct 21, 2016 at 3:29 AM, Steven D'Aprano wrote:
> As for the amount of good, this proposal originally came from PyPy.
Just to be clear, I'm not a PyPy dev, and the PyPy devs' contribution
here was mostly to look over a draft I circulated and to agree that it
seemed like something that'd b
On Fri, Oct 21, 2016 at 3:48 PM, Amit Green wrote:
> NOTE: This is my first post to this mailing list, I'm not really sure
> how to post a message, so I'm attempting a reply-all.
>
> I like Nathaniel's idea for __iterclose__.
>
> I suggest the following changes to deal with a few of the comp
On 10/21/2016 03:48 PM, Amit Green wrote:
NOTE: This is my first post to this mailing list, I'm not really sure
how to post a message, so I'm attempting a reply-all.
Seems to have worked! :)
I like Nathaniel's idea for __iterclose__.
I suggest the following changes to deal with a few
NOTE: This is my first post to this mailing list, I'm not really sure
how to post a message, so I'm attempting a reply-all.
I like Nathaniel's idea for __iterclose__.
I suggest the following changes to deal with a few of the complex issues
he discussed.
1. Missing __iterclose__, or a valu
On 21 October 2016 at 21:59, Chris Barker wrote:
>> So (it seems to
>> me) that you're talking about changing the behaviour of for-loops to
>> suit only a small proportion of cases: maybe 10% of 10%.
>
>
> I don't see what the big overhead is here. for loops would get a new
> feature, but it would
On Fri, Oct 21, 2016 at 12:12 AM, Steven D'Aprano
wrote:
> Portability across Pythons... if all Pythons performed exactly the same,
> why would we need multiple implementations? The way I see it,
> non-deterministic cleanup is the cost you pay for a non-reference
> counting implementation, for th
On 2016-10-21 11:19 AM, Gustavo Carneiro wrote:
Personally, I hadn't realised we had this problem in asyncio until now.
Does this problem happen in asyncio at all? Or does asyncio somehow work
around it by making sure to always explicitly destroy the frames of all
coroutine objects, as long a
Le 21/10/16 à 14:35, Paul Moore a écrit :
[1] As I understand it. CPython's refcounting GC makes this a
non-issue, correct?
Wrong. Any guarantee that you think the CPython GC provides goes out of
the window as soon as you have a reference cycle. Refcounting does not
actually make GC determin
Personally, I hadn't realised we had this problem in asyncio until now.
Does this problem happen in asyncio at all? Or does asyncio somehow work
around it by making sure to always explicitly destroy the frames of all
coroutine objects, as long as someone waits on each task?
On 21 October 2016 at
On 2016-10-21 7:13 AM, Steven D'Aprano wrote:
Consistency is over-rated, and we already have inconsistency
here: file iterators behave differently from list iterators, because
they can be closed:
This is **very** arguable :)
Yury
___
Python-ideas m
On 2016-10-21 6:29 AM, Steven D'Aprano wrote:
On Wed, Oct 19, 2016 at 05:52:34PM -0400, Yury Selivanov wrote:
[..]
With you proposal, to achieve the same (and make the code compatible
with new for-loop semantics), users will have to implement both
__iterclose__ and __del__.
As I ask above, c
On 21 October 2016 at 12:23, Steven D'Aprano wrote:
> On Fri, Oct 21, 2016 at 11:03:51AM +0100, Paul Moore wrote:
>
>> At the moment, the take home message for such users feels like it's
>> "you might need to scatter preserve() around your code, to avoid the
>> behaviour change described above, wh
On Fri, Oct 21, 2016 at 11:07:46AM +0100, Paul Moore wrote:
> On 21 October 2016 at 10:53, Steven D'Aprano wrote:
> > On Wed, Oct 19, 2016 at 12:33:57PM -0700, Nathaniel Smith wrote:
> >
> >> I should also say, regarding your specific example, I guess it's an
> >> open question whether we would wa
On Fri, Oct 21, 2016 at 11:03:51AM +0100, Paul Moore wrote:
> At the moment, the take home message for such users feels like it's
> "you might need to scatter preserve() around your code, to avoid the
> behaviour change described above, which you glazed over because it
> talked about all that coro
On Wed, Oct 19, 2016 at 05:52:34PM -0400, Yury Selivanov wrote:
> IOW I'm not convinced that if we implement your proposal we'll fix 90%
> (or even 30%) of cases where non-deterministic and postponed cleanup is
> harmful.
Just because something doesn't solve ALL problems doesn't mean it isn't
On 21 October 2016 at 10:53, Steven D'Aprano wrote:
> On Wed, Oct 19, 2016 at 12:33:57PM -0700, Nathaniel Smith wrote:
>
>> I should also say, regarding your specific example, I guess it's an
>> open question whether we would want list_iterator.__iterclose__ to
>> actually do anything. It could fl
On 21 October 2016 at 07:03, Nathaniel Smith wrote:
> Oh, goodness, no -- like Yury said, the use cases here are not
> specific to async at all. I mean, none of the examples are async even
> :-).
[...]
Ah I follow now. Sorry for the misunderstanding, I'd skimmed a bit
more than I realised I had.
You know, I'm actually starting to lean towards this proposal and away
from my earlier objections...
On Wed, Oct 19, 2016 at 12:33:57PM -0700, Nathaniel Smith wrote:
> I should also say, regarding your specific example, I guess it's an
> open question whether we would want list_iterator.__itercl
On Thu, Oct 20, 2016 at 11:03:11PM -0700, Nathaniel Smith wrote:
> The motivation here is that prompt (non-GC-dependent) cleanup is a
> good thing for a variety of reasons: determinism, portability across
> Python implementations, proper exception propagation, etc. async does
> add yet another ent
On Wed, Oct 19, 2016 at 7:07 PM, Terry Reedy wrote:
> On 10/19/2016 12:38 AM, Nathaniel Smith wrote:
>
>> I'd like to propose that Python's iterator protocol be enhanced to add
>> a first-class notion of completion / cleanup.
>
>
> With respect the the standard iterator protocol, a very solid -1 f
On Wed, Oct 19, 2016 at 3:07 PM, Paul Moore wrote:
> On 19 October 2016 at 20:21, Nathaniel Smith wrote:
>> On Wed, Oct 19, 2016 at 11:38 AM, Paul Moore wrote:
>>> On 19 October 2016 at 19:13, Chris Angelico wrote:
Now it *won't* correctly call the end-of-iteration function, because
t
On 10/19/2016 12:38 AM, Nathaniel Smith wrote:
I'd like to propose that Python's iterator protocol be enhanced to add
a first-class notion of completion / cleanup.
With respect the the standard iterator protocol, a very solid -1 from
me. (I leave commenting specifically on __aiterclose__ to
On 2016-10-19 6:07 PM, Paul Moore wrote:
I missed that you propose phasing this in, but it doesn't really alter
much, I think the current behaviour is valuable and common, and I'm -1
on breaking it. It's just too much of a fundamental change to how
loops and iterators interact for me to be comf
Hey Nathaniel - I like the intent here, but I think perhaps it would
be better if the problem is approached differently.
Seems to me that making *generators* have a special 'you are done now'
interface is special casing, which usually makes things harder to
learn and predict; and that more the net
On 19 October 2016 at 20:21, Nathaniel Smith wrote:
> On Wed, Oct 19, 2016 at 11:38 AM, Paul Moore wrote:
>> On 19 October 2016 at 19:13, Chris Angelico wrote:
>>> Now it *won't* correctly call the end-of-iteration function, because
>>> there's no 'for' loop. This is going to either (a) require
On Wed, Oct 19, 2016 at 11:13 AM, Chris Angelico wrote:
> On Thu, Oct 20, 2016 at 3:38 AM, Random832 wrote:
>> On Wed, Oct 19, 2016, at 11:51, Yury Selivanov wrote:
>>> I'm -1 on the idea. Here's why:
>>>
>>>
>>> 1. Python is a very dynamic language with GC and that is one of its
>>> fundamental
Nathaniel,
On 2016-10-19 5:02 PM, Nathaniel Smith wrote:
Hi Yury,
Thanks for the detailed comments! Replies inline below.
NP!
On Wed, Oct 19, 2016 at 8:51 AM, Yury Selivanov wrote:
I'm -1 on the idea. Here's why:
1. Python is a very dynamic language with GC and that is one of its
fun
On Wed, Oct 19, 2016 at 1:33 PM, Yury Selivanov wrote:
> On 2016-10-19 3:33 PM, Nathaniel Smith wrote:
>
>>> lst = [1,2,3,4]
>>> >it = iter(lst)
>>> >for i in it:
>>... if i == 2: break
>>
>>>
>>> >for i in it:
>>... print(i)
>>3
>>
Ohhh, sorry, you want __iterclose__ to happen when iteration is terminated
by a break statement as well? Okay, I understand, and that's fair.
However, I would rather that people be explicit about when they're
iterating (use the iteration protocol) and when they're managing a resource
(use a c
Hi Yury,
Thanks for the detailed comments! Replies inline below.
On Wed, Oct 19, 2016 at 8:51 AM, Yury Selivanov wrote:
> I'm -1 on the idea. Here's why:
>
>
> 1. Python is a very dynamic language with GC and that is one of its
> fundamental properties. This proposal might make GC of iterators
On 2016-10-19 3:33 PM, Nathaniel Smith wrote:
lst = [1,2,3,4]
>it = iter(lst)
>for i in it:
>>... if i == 2: break
>>
>for i in it:
>>... print(i)
>>3
>>4
>
>>
>>With the proposed behaviour, if I understand it, "it" would be closed
>>after the first loop, so resuming "it
On Thu, Oct 20, 2016 at 7:14 AM, Neil Girdhar wrote:
> class AddIterclose:
>
> def __init__(self, iterable, iterclose):
> self.iterable = iterable
> self.iterclose = iterclose
>
> def __iter__(self):
> try:
> for x in self.iterable:
> yie
On Wed, Oct 19, 2016 at 2:11 PM Nathaniel Smith wrote:
> On Wed, Oct 19, 2016 at 10:08 AM, Neil Girdhar
> wrote:
> >
> >
> > On Wed, Oct 19, 2016 at 11:08 AM Todd wrote:
> >>
> >> On Wed, Oct 19, 2016 at 3:38 AM, Neil Girdhar
> >> wrote:
> >>>
> >>> This is a very interesting proposal. I just
On 2016-10-19 12:21, Nathaniel Smith wrote:
>Also, unless I'm misunderstanding the proposal, there's a fairly major
>compatibility break. At present we have:
>
lst = [1,2,3,4]
it = iter(lst)
for i in it:
>... if i == 2: break
>
for i in it:
>... print(i)
>3
>4
>
>Wi
On Wed, Oct 19, 2016 at 12:21 PM, Nathaniel Smith wrote:
> On Wed, Oct 19, 2016 at 11:38 AM, Paul Moore wrote:
>> On 19 October 2016 at 19:13, Chris Angelico wrote:
>>> Now it *won't* correctly call the end-of-iteration function, because
>>> there's no 'for' loop. This is going to either (a) req
On Wed, Oct 19, 2016 at 11:38 AM, Paul Moore wrote:
> On 19 October 2016 at 19:13, Chris Angelico wrote:
>> Now it *won't* correctly call the end-of-iteration function, because
>> there's no 'for' loop. This is going to either (a) require that EVERY
>> consumer of an iterator follow this new prot
On Wed, Oct 19, 2016 at 2:38 PM, Paul Moore wrote:
> On 19 October 2016 at 19:13, Chris Angelico wrote:
> > Now it *won't* correctly call the end-of-iteration function, because
> > there's no 'for' loop. This is going to either (a) require that EVERY
> > consumer of an iterator follow this new p
On 10/19/2016 11:38 AM, Paul Moore wrote:
Also, unless I'm misunderstanding the proposal, there's a fairly major
compatibility break. At present we have:
lst = [1,2,3,4]
it = iter(lst)
for i in it:
... if i == 2: break
for i in it:
... print(i)
3
4
With the proposed behaviour, if I
On 19 October 2016 at 19:13, Chris Angelico wrote:
> Now it *won't* correctly call the end-of-iteration function, because
> there's no 'for' loop. This is going to either (a) require that EVERY
> consumer of an iterator follow this new protocol, or (b) introduce a
> ton of edge cases.
Also, unles
On Thu, Oct 20, 2016 at 3:38 AM, Random832 wrote:
> On Wed, Oct 19, 2016, at 11:51, Yury Selivanov wrote:
>> I'm -1 on the idea. Here's why:
>>
>>
>> 1. Python is a very dynamic language with GC and that is one of its
>> fundamental properties. This proposal might make GC of iterators more
>> de
On Wed, Oct 19, 2016 at 10:08 AM, Neil Girdhar wrote:
>
>
> On Wed, Oct 19, 2016 at 11:08 AM Todd wrote:
>>
>> On Wed, Oct 19, 2016 at 3:38 AM, Neil Girdhar
>> wrote:
>>>
>>> This is a very interesting proposal. I just wanted to share something I
>>> found in my quick search:
>>>
>>>
>>> http:/
On Wed, Oct 19, 2016 at 11:08 AM Todd wrote:
> On Wed, Oct 19, 2016 at 3:38 AM, Neil Girdhar
> wrote:
>
> This is a very interesting proposal. I just wanted to share something I
> found in my quick search:
>
>
> http://stackoverflow.com/questions/14797930/python-custom-iterator-close-a-file-on-
On 2016-10-19 12:38 PM, Random832 wrote:
On Wed, Oct 19, 2016, at 11:51, Yury Selivanov wrote:
I'm -1 on the idea. Here's why:
1. Python is a very dynamic language with GC and that is one of its
fundamental properties. This proposal might make GC of iterators more
deterministic, but that is
On Wed, Oct 19, 2016, at 11:51, Yury Selivanov wrote:
> I'm -1 on the idea. Here's why:
>
>
> 1. Python is a very dynamic language with GC and that is one of its
> fundamental properties. This proposal might make GC of iterators more
> deterministic, but that is only one case.
There is a hug
I'm -1 on the idea. Here's why:
1. Python is a very dynamic language with GC and that is one of its
fundamental properties. This proposal might make GC of iterators more
deterministic, but that is only one case.
For instance, in some places in asyncio source code we have statements
like t
On Wed, Oct 19, 2016 at 3:38 AM, Neil Girdhar wrote:
> This is a very interesting proposal. I just wanted to share something I
> found in my quick search:
>
> http://stackoverflow.com/questions/14797930/python-
> custom-iterator-close-a-file-on-stopiteration
>
> Could you explain why the accepte
This is a very interesting proposal. I just wanted to share something I
found in my quick search:
http://stackoverflow.com/questions/14797930/python-custom-iterator-close-a-file-on-stopiteration
Could you explain why the accepted answer there doesn't address this issue?
class Parse(object):
On 19 October 2016 at 12:33, Oscar Benjamin wrote:
>
>> New convenience functions
>> -
>>
>> The ``itertools`` module gains a new iterator wrapper that can be used
>> to selectively disable the new ``__iterclose__`` behavior::
>>
>> # XX FIXME: I feel like there might be
On 17 October 2016 at 09:08, Nathaniel Smith wrote:
> Hi all,
Hi Nathaniel. I'm just reposting what I wrote on pypy-dev (as
requested) but under the assumption that you didn't substantially
alter your draft - I apologise if some of the quoted text below has
already been edited.
> Always inject r
Thanks Nathaniel for this great proposal.
As I went through your mail, I realized all the comments I wanted to
make were already covered in later paragraphs. And I don't think there's
a single point I disagree with.
I don't have a strong opinion about the synchronous part of the
proposal. I
Hi all,
I'd like to propose that Python's iterator protocol be enhanced to add
a first-class notion of completion / cleanup.
This is mostly motivated by thinking about the issues around async
generators and cleanup. Unfortunately even though PEP 525 was accepted
I found myself unable to stop pond
67 matches
Mail list logo
